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Siiiamary;  To  what  kinds  of  events  can  a 
McCulloch— Pitts  nerve  net  respond  by  firing 
a  certain  neuron?  More  generally,  to  what 
kinds  of  events  can  any  finite  automaton 
respond  by  assuming  one  of  certain  states? 
This  memorandum  Is  devoted  to  an  elementary 
exposition  of  the  problems  and  of  results 
obtained  on  It  during  investigations  In 
August  1951* 


REPRESENTATION  OF  EVENTS 
IN  NERVE  NETS  AND  FINITE  AUTOMATA 
S.  C.  Kleene 


INTRODUCTION; 

1.  Stimulus  and  Response:  An  organism  or  robot  receives 
certain  stimuli  (via  its  sensory  receptor  organs)  and  performs 
-certain  actions  (via  its  effe’ctor  organs).  To  say  that  certain 
actions  are  a  response  to  certain  stimuli  means,  in  the  simplest 
case,  that  the  actions  are  performed  when  those  stimuli  occur 
and  not  when  they  do  not  occur. 

Since  both  the  stimuli  and  the  actions  may  be  very  com¬ 
plicated,  the  relationship  between  the  two  is  very  complicated. 

In  order  to  simplify  our  analysis,  we  may  leave  out  of 
account  the  complexities  of  the  response.  To  do  this,  we  reason 
that  any  kind  of  stimulation,  or  briefly,  any  event  which  affects 
action,  in  the  sense  that  according  as  the  event  occurs  or  does 
not,  under  some  set  of  other  circumstances  held  fixed,  a  differ¬ 
ent  action  ensues,  must  have  a  representation  In  the  state  of 
the  organism  or  machine,  after  the  event  has  occurred  and  prior 
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to  the  ensuing  action  (which  action  may  ciepend  on  the  occur¬ 
rence  of  many  other  events). 

We  then  ask  what  kinds  of  events  are  capable  of  being 
represented  in  the  state  of  the  organism  or  machine. 

We  shall  see  later  (Section  5.5)  that  there  is  no  loss  of 
generality  in  considering  the  representation,  in  the  case  of 
■  nerve  nets,  to  have  the  simple  form  of  the  firing  (or  sometimes 
the  non— firing  Instead)  at  a  certain  time  of  a  certain  neuron. 

For  explaining  response  as  due  to  stimulus,  it  would  then 
remain  to  assemble  the  complicated  molar  response  out  of  these 
molecular  representations  of  molar  stimuli. 

In  this  remaining  problem,  it  could  make  a  greftt  difference 
what  events  are  selected  for  molecular  representation,  as  the 
abstract  from  experience  which  is  to  form  the  basis  of  action. 

However,  we  shall  not  enter  into  this  here,  except  as  it 
reflects  on  the  problem  of  representing  events;  nor  shall  we 
enter  into  the  analogies  between  the  analysis  just  described 
and  the  psychological  phenomena  in  which  raw  sense  data  lead 
through  percepts  and  concepts  to  overt  behavior. 

0 

2.  Nerve  Nets  and  Behavior;  McCulloch  and  Pitts  (19^3) 
in  their  fundamental  paper  on  the  logical  analysis  of  nervous 
activity  formulated  certain  assumptions  which  we  shall  recapi¬ 
tulate  below  (Section  3)* 

These  assumptions  are  an  abstraction  from  the  data  which 
neurophysiology  provides.  The  abstraction  gives  a  model,  in 
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terms  of  which  it  becomes  an  exact  mathematical  problem  to  see 
what  kinds  of  behavior  the  model  can  explain.  The  question  is 
left  open  how  closely  the  model  describes  the  activity  of  actual 
nerve  nets;  and  some  modifications  in  the  assumptions  lead  to 
similar  models.  Neurophysiology  does  not  currently  say  which 
of  these  models  is  most  nearly  correct — ^it  is  not  plausible  that 
any  one  of  them  fits  exactly.  It  is  noteworthy,  however,  that 
one  of  McCulloch  and  Pitts'  results  is  that  these  several  other 
models  are  capable  of  producing  only  the  same  behavior  as  the 
first  one. 

Until  neuro-physiology  tells  us  more  about  the  actual 
process,  it  is  instructive  to  see  what  behavior  the  model 
admits.  Our  results  are  to  the  effect  that  "it  could  be  this 
way,  and  quite  possibly  the  real  process  is  significantly  simi¬ 
lar  to  this.”  Furthermore,  such  studies  have  applications  in 
robotology,  when  we  wish  to  describe  on  paper  (or  build  in  the 
metal,  using  elements  which  behave  like  McCulloch— Pitts  neurons) 
a  robot  to  behave  in  a  pre— assigned  manner. 

This  study  can  be  pursued  on  two  levels,  a  strictly 
practical  one  and  a  theoretical  one.  On  the  former,  we  are 
concerned  with  constructing  particular  nerve  nets  to  give  par¬ 
ticular  described  behavior;  in  the  latter,  we  develop  general 
methods  for  constructing  nets  to  give  behavior,  and  Investigate 
the  limitations  within  which  this  is  possible. 

This  memorandum  deals  with  studies  on  the  second  level,  but 
actually  the  two  are  not  clearly  separated.  The  general  methods 
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may  be  practical  or  suggest  methods  which  are,  and  the  inves¬ 
tigations  of  the  limitations  may  contribute  better  understanding 
of  the  problems  which  are  faced  on  the  practical  level. 

McCulloch  and  Pitts  give  such  a  theoretical  investiga¬ 
tion,  consisting  of  a  theory  for  nerve  nets  without  "circles” 
(Part  II  of  their  paper)  and  a  theory  for  arbitrary  nerve  nets 
(Part  III).  The  present  memorandum  is  partly  an  exposition  of 
the  McCulloch— Pitts  results;  but  we  found  the  part  of  their 
paper  which  treats  of  arbitrary  nets  obscure;  so  we  have  pro¬ 
ceeded  ihdependently  here. 

Under  the  McCulloch— Pitts  assumption  of  the  all— orwio thing 
character  of  a  neuron's  firing  (which  is  close  to  the  biological 
reality)  and  their  assumption  which  quantizes  time  so  that  all 
neurons  have  their  moments  of  possible  firing  in  phase,  a  nerve 
net  has  the  character  of  a  digital  automaton.  Here  we  are  using 
"digital"  in  contrast  to  "analog,"  in  the  sense  familiar  in 
connection  with  computing  machines. 

It  seems  quite  clear  that  many  physical  processes  of 
control  are  partly  analog  in  character.  For  example,  the  res¬ 
piratory  cycle  of  activity  can  be  controlled  consciously (by 
nervous  means,  which  are  digital);  but  most  of  the  time  it  is 
regulated  by  a  nervous  response  in  the  respiratory  center  of 
the  hrain  to  the  carbon— dioxide  level  in  the  blood  (an  analog 
quantity) . 

Just  as  in  mathematics  continuous  processes  can  be  approxl— 

Kated  by  discrste  orea^  it  is  plausible  that  ar.y  anairg  alemerta 
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In  bodily  control  could  be  approximated  in  their  effect  by 
digital  ones.  Nevertheless,  the  analog  or  partly  analog  con¬ 
trols  may  remain  the  simplest  and  most  efficient. 

One  of  the  results  of  systematic  theoretical  investiga¬ 
tions  of  the  potentialities  of  digital  control  might  be  to 
demonstrate  that  other  principles,  e.g.,  analog  mechanisms  or 
the  introduction  of  random  inputs,  may  be  necessary  to  produce, 
or  to  produce  economically,  certain  kinds  of  behavior. 

Another  tacit  assumption  of  the  present  mathematical  theory 
is  that  there  are  no  errors  in  the  functioning  of  neurons;  l.e., 
a  given  neuron  fires  at  a  given  moment,  if  and  only  if  it  shouM 
under  the  McCulloch— Pitts  rules.  Of  course,  this  is  unrealistic, 
either  for  living  neurons  or  for  the  equivalent  units  of  a 
mechanical  automaton.  It  seems  natural,  however,  to  build  a 
theory  of  what  happens  assuming  no  malfunctioning,.  In  this 
theory,  we  represent  the  occurrence  of  an  event  by  the  firing  of 
a  single  neuron.  Biologically,  it  is  implausible  that  impor¬ 
tant  information  should  ever  be  represented  in  an  organism  in 
this  way.  But  by  duplications  of  nets  (many  processes  being 
carried  out  in  parallel  circuits),  one  could  expect  then  to 
secure  the  same  results  with  small  probability  of  failure  in 
nets  constructed  of  fallible  neurons. 

Returning  to  the  formulation  of  the  problem  as  given  in 
Sect.  1,  we  shall  now  in  Part  I  show  that  all  events  of  a 
certain  class  can  be- represented  by  the  firing  (or  in  some  cases, 
the  non— firing)  of  a  certain  neuron.  The  discussion  of  the 
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converse  Is  left  mainly  to  Part  II,  where  we  generalize  to 
representabllity  In  any  finite  digital  automaton. 

PART  I  —  NERVE  NETS; 

McCulloch— Pitts  Nerve  Nets;  Under  the  assxiraptions  of 
McCulloch  and  Pitts  (19^3) ^  ^  nerve  cell  or  neuron  consists  of 
a  soma,  whence  nerve  fibers  (axons)  lead  to  one  or  more  end bulbs . 

A  nerve  net  Is  an  arrangement  of  a  finite  number  of  neurons. 
In  which  each  endbulb  of  any  neuron  is  adjacent  to  the  soma  of 
not  more  than  one  neuron  (the  same  or  another);  the  separating 
gap  is  a  synapse .  Each  endbulb  is  either  excitatory  or  Inhibitory 
(not  both) . 

We  call  the  neurons  (zero  or  more)  on  which  no  endbulbs 
Impinge  Input  neurons;  the  others.  Inner  neurons.  (McCulloch 
and  Pitts  say  "peripheral  afferent  neurons"  for  the  former,  but  ‘ 
It  Is  convenient  to  have  a  shorter  phrase.  "Efferent  neurons" 
might  be  used  for  the  latter,  but  it  Is  not  clear  to  us  that 
this  Is  appropriate.  (As  the  present  paper  is  only  a  working 
paper,  we  welcome  suggestions  as  to  Improvements  In  the  termi¬ 
nology,  ) 

At  equally  separated  moments  .of  time  (which  we  taOce  as  the 
Integers  on  a  time  scale,  the  same  for  all  neurons  In  a  given 
net),  each  neuron  of  the  net  Is  capable  of  firing  or  not  firing 
(being  quiet)  In  an  all-or-nothing  manner.  For  an  Input  neuron, 
the  firing  or  non— firing  at  any  time  ^  Is  determined  by  condi¬ 
tions  outside  the  net.  One  can  suppose  each  Is  Impinged  upon 
by  a  sensory  receptor  organ,  which  under  suitable  conditions  In 
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the  environment  causes  the  neuron  to  fire  at  time  t.  For  an 
Inner  neuron,  the  condition  for  firing  at  time  t  Is  that  at  least 
a  certain  number  h  {the  threshold  of  that  neuron)  of  the  exci¬ 
tatory  endbulbs,  and  none  of  the  Inhibitory  endbulbs,  synapslng 
on  It  belong  to  neurons  which  fired  at  time  Wl. 

For  Illustration,  consider  the  following  nerve  net,  with 
Input  neurons  J,  K,  L,  M,  and  N  and  Inner  neuron  P.  Excitatory 
endbulbs  are  shown  as  dots,  and  Inhibitory  as  circles.  The 
threshold  of  P  Is  3  as  shown  by  the  figure  on  the  triangle  repre¬ 
senting  Its  soma.  The  fomula  written  below  the  net  expresses  • 


P(t)  =  J(;^l)  g.  K(;^l)  £  L(t^l)  Ic  F(pry  2c  M ( W ) 

1 

In  logical  symbolism  that  neuron  P  fires  at  time  t  |in  the 
symbols,  ”l(t)"J,  If  and  only  If  (in  symbols,  "H”)  all  of 
J,  K,  and  L  and  none  of  M  and  N  fire  at  time  ^l.("  means 
."and,”  and  "  "  means  "not"). 

The  method  of  nerve  net  construction  Illustrated  In  Fig.  1 
applies  for  any  niamber  >  1  of  unnegated  propositions  (3  In 
Fig.  l)  and  any  number  >  0  of  negated  propositions  (2  In  Fig.  l) 
combined  conjunctively. 
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Two  other  nets  (Figs.  2  and  3)  illustrate  additional  methttas 
which  will  be  used  in  nerve  net  constructions  in  Sect.  5* 


Here  "V"  means  "or"  (in  the  non— exclusive  sense). 

4.  The  Input  to  a  Nerve  Net;  Consider  a  nerve  net  with  k 
input  neurons  Ni,...,Nj^  .  The  Input  (or  experience)  over  all 

past  time  up  to  the  present  moment  Inclusive  can  be  represented 
by  a  table  or  matrix  with  k  columns  corresponding  to  the  input 
neurons,  and  with  rows  corresponding  to  the  moments  counting 
backward  from  the  present  moment  _t  •»>  £.  The  positions  are  filled 
with  O's  and  I's,  where  0  is  to  stand  for  quiescence,  and  1  for 
firing,  of  the  neuron  in  question  at  the  time  in  question. 

For  example,  with  k  »!*  2  the  matrix  might  be  as  follows: 
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t  Ni  Nz 

p  1  0 

2-111 
2-2  0  1 

2r^  1  1 


Fig,  i 

The  1  in  the  first  row  and  first  column  means  that  Ni  fired 
at  time  2J  the  0  in  the  first  row  and  second  column  that  Na  did 
not  fire  at  time  £;  the  1  in  the  second  row  and  first  column  that 
Ni  fired  at  time  £”1^  etc. 

If  this  table  is  extended  down  infinitely,  we  have  a  repre¬ 
sentation  of  the  input,  thought  of  as  extending  over  all  past 
time.  The  discussion  whether  we  should  think  of  past  time  as 
infinite  will  be  left  to  the  place  where  it  becomes  crucial 
(Sect.  6.1).  For  the  purposes  of  Sect.  5  we  need  merely  assume 
that  it  extends  back  in  each  case  as  far  as  the  number  of  rows 
of  the  matrix  being  considered  there. 

By  an  event  we  mean  any  property  of  the  input.  Thus,  any 
subclass  of  the  class  of  all  the  possible  tables  represents  an 
event,  which  occurs  when  the  table  describing  the  actual  input 
belongs  to  this  subclass.  In  coin  tossing  or  dice  throwing, 
examples  of  events  are  "heads”  or  "eleven"  (as  sum  of  the 
numbers  of  spots  on  the  uppermost  faces  of  the  two  dice).  Here 
examples  are:  (l)  Ni  fired  at  time  £.  (2)  N2  did  not  fire 


RM-704 
Page  10 


at  time  £,  and  Ni  fired  at  time  £-1.  (3)  One  of  Ni  and  Na 

fired  at  time  £*  Ni  and  Na  both  fired  at  time  £.  (5)  Na 

fired  at  some  time.  (6)  Na  fired  at  every  time  except  £. 

Of  these,  the  (present  and)  past  described  by  the  table  in 
Fig.  4  constitutes  an  occurrence  of  events  (l),  (2),  (3),  and 
(5),  but  not  of  (4),  while  we  need  to  know  the  rest  of  the  table 
to  see  whether  it  constitutes  an  occurrence  of  (6). 


5.  Definite  Events; 

5.1.  "Definite  events”  defined:  We  shall  first  restrict 
ourselves  to  events  whfch  refer  to  a  fixed  period  of  time,  con¬ 
sisting  of  the  ^  (>  i)  moments  £-^+l,...,£  ending  with  the 
present.  This  means  that  in  any  table  such  as  that  of  Fig.  4 
we  consider  only  the  uppermost  ^  rows;  e.g.,  with  ^  =  5: 


t  Ni  Na 

£  1  0 

p-1  1  1 

£-2  0  1 


Ni  {£)  is  (e) 

%  Ni(£-1)  ^  Na(£-1) 
1  iis(£-2). 


Fig.  5 


The  formula  at  the  right  expresses  the  same  as  is  expressed  by 
the  table;  l.e.,  it  says  that  Nx  fires  at  time  £  (”Ni(£)")  and 
("^")  Na  does  not  fire  at  time  £  ( "Ma (£) " ) ,  and  Nx  fires  at 
time  £-1  ("Nx(£-1)"),  etc. 

We  call  an  event  referring  to  Just  these  ^  moments  definite 
of  length  (or  duration)  With  k  input  neur*on3,  there  are 
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exactly  ^  entries  in  a  table  describing  the  input  for  these 
moments.  Therefore,  there  are  exactly  2^  possible  such  tables. 
Therefore,  there  are  exactly  2®  definite  events  of  length  ^ 
since  any  particular  event  (of  length  ^  with  k  input  neurons) 

Is  obtained  by  saying  which  of  the  2^^  tables  would  constitute 
(if  they  represented  the  actual  past)  an  occurrence  of  the  everlt. 

We  call  an  event  positive.  If  it  only  occurs  when  at  least 
one  input  neuron  fires  during  the  period  to  which  the  event 
refers.  There  are  2®  ;  '  ‘  definite  positive  events,  since  now 
we  exclude  as  an  occurrence  of  the  event  that  past  described 
by  the  table  of  all  O's. 

5*^  ^sflnite  posj.tlvs 

H.  1 

Theorem  1 .  To  each  of  the  2^  definite  gositive 

events  €>f  lenfth  (wlt%  k  Input:  neurons ] ,  there  Is  a  iterve  net 
having  an  inner  neuron  which  fires  at  tltne  £+2,  if  and  only  if 
the  event  occurs  during  time  £-J^+l  ^ 

This  theorem,  except  for  the  remark  that  the  "lag"  can  be 
held  to  2,  is  given  by  McCulloch  and  Pitts  (19^5)* 

Proof:  To  Illustrate,  say  the  event  is  one  which  occurs 
if  and  only  if  the  pattern  of  firings  over  the  past  is  represertted 
either  by  the  table  of  Fig.  5  or  by  the  following  table: 


t  Ni  Na 


£ 


1 


0 


£-1 

£-2 


1 


0 

0 


Ni(E)2r^TET 
^  N 1  (Pr^ )  Na(2r^J 

^  N 1  (g— 2 )  ^  Na  (g-^  J . 


1 
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That  is,  just  these  two  (out  of  the  2®’®  =  64)  tables  are  to 
constitute  an  occurrence  of  the  event.  The  event  is  described 
by  the  following  logical  formula: 

j^i  (£.)  ^  Na  (£^)  ^  Ni  (£7-1 )  S'  Ng  (p^-l )  S’  (g— 27*S^  Ng  (£^2  )J 

V  [^.(E)2-S5W2.!i.(e-i)!j-5JTFr)2N.(E-2)2.^TF^’ 

Figure  7_ 

This  formula  is  a  "disjunction”  having  two  "members"  or  "terms," 
each  of  vdiich  is  a  "conjunction"  having  six  "members"  or 
"factors."  The  two  terms  correspond  to  the  tables  of  Pigs.  5 
and  6,  respectively. 

A  nerve  net  which  represents  the  event  with  lag  2  is  con¬ 
structed  as  follows: 


Figure  8 
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Using  the  method  illustrated  in  Fig. 

N 1  (£)  =  N 1  (£-1 ) ,  N^2  (£)  ~  Na  (£-l) , 

Ni  (p)  =  iii  (p-2)  j  N2(£)  =  Na  (£-8)  . 

Now  using  the  device  of  Fig.  1, 

Mi{£fl)sNx(£)  Ir  Ni(£)  ^  IfW  ^  Ns  (r) 

=  N 1  (£)  ^  ^a  (£)  ^  N 1  (£—1 )  <^  Na  (£—1 )  ^  Na  (Et^ )  > 

i.e..  Ml  fires  at  time  £fl  if  and  only  if  the  past  is  described 
by  the  table  of  Fig.  5  (or  the  first  conjunction  in  Fig.  7). 
Likewise,  the  firing  of  Ma  at  £+1  corresponds  to  the  table  of 
Fig.  6  (or  the  second  conjunction  in  Pig.  7).  Finally,  by  the 
method  of  Fig.  2, 

P(£+2)=  Mi(£+1)  V  M2(E+i)- 

Combining  this  with  what  has  already  been  remarked,  P  fires  at 
£+2  if  and  only  if.  the  event  occurs  during  time  £-2  to  £. 

Th'e  method  of  the  illustration  applies  to  every  definite 
positive  event  which  occurs  for  some  one  or  more  tables.  By 
the  restriction  that  the  event  be  positive,  each  table  must  have 
at  least  one  1  in  it,  which  assures  the  applicability  of  the 
device  of  Fig.  1. 

There  remains  the  case  of  the  event  which  never  occurs. 

This  is  represented,  e.g.,  by  the  following  net: 

P  never  fires  at  time  £+2 
(or  in  symbols,  e.g., 

P  (£+2 )  —  N 1  (£)  If  N 1  (£)1 . 

Figure  2 


vv 

Ni  Na 
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5.3  Simpler  nerve  nets ;  While  this  proves  the  theorem. 

It  Is  to  be  observed  that  often  much  simpler  nets  can  be  con¬ 
structed  than  that  given  by  the  above  method  of  proving  the 
theorem. 

Readers  having  technical  acquaintance  with  symbolic  logic 
will  recognize  that  the  construction  used  in  proving  the  theorem 
corresponds  to  the  principal  disjunctive  normal  fonn  of  Hilbert— 
Ackermann  (1928)  which  describes  the  event.  In  the  illustration, 
the  normal  form  is  the  formula  of  Pig.  7.  Each  of  the  tables 
which  describe  an  occurrence  Of  the  event  is  represented  by  a 
conjunction  or  term  in  the  normal  form  and  is  taken  care  of 
separately  in  building  the  nerve  net.  This  makes  the  proof  of 
the  theorem  simple,  but  the  net  complicated. 

Consider  for  example  the  event  which  is  described  by  saying 
that  the  table  must  be  of  one  of  the  two  following  forms,  where 
either  a  0  or  a  1  can  be  supplied  independently  for  each  blank 

tt  It 


t  Ni  N2 


t  Ni  Na 


£ 

£-1 

£“2 


0 


£ 


1 


or 


p-1  — 

2-2  — 


liW  &  N;2(£r2]]  VN2(£r-l)- 


Figure  10 
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In  terms  of  complete  tables,  this  event  would  be  expressed 
by  a  choice  between  2 V2^— 2^=40  tables,  Including  that  of  Fig.  5 
as  one  of  them.  The  principal  disjunctive  normal  form  would  be 
a  disjunction  df  40  conjunctions.  The  simple  formula  shown  in 
Fig.  10  which  represents  it  is  a  disjunctive  normal  form  (not  a 
principal  one).  The  event  is  represented  in  the  sense  of  the 
theorem  by  the  following  net. 


In  this  net  we  show  only  Ng  as  an  input  neuron,  although 
we  defined  our  events  in  terms  of  two  input  neurons  Ni  and  Ng 
in  our  Illustrations.  The  net  of  Fig.  11  can  constitute  a  part 
of  a  larger  net  having  Ni  also  as  input  neuron,  entering  in  such 
a  way  that  it  has  no  endbulbs  on  any  of  the  neurons  shown  in 
Fig.  11.  The  example  Illustrates  that  if  we  begin  by  defining 
events  relative  to  a  set  Ni,..i.,Nj^  of  input  neurons,  we  need 

actually  use  in  our  net  constructions  only  those  of  Ni,...,N^ 
whose  firing  or  non— firing  affects  whether  the  event  occurs. 

There  is  a.  corresponding  treatment,  with  the  same  lag,  for 
conjunctive  normal  forms.  We  begin  by  considering  the  following 
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j_ jL ^  ^Ci  wlij-Ci*  nOjrnio.a.  x&  s.  p'i.^^iiC^|>^x  w*iS 


Figure  12 


To  see  that  this  works,  observe  that  we  use  Fig.  2  In  obtaining 
Ml,  so  that 

Mi(2;M)=  Ni(£)  V  N2(£)\/N3(£); 

but  Fig.  1  to  obtain  Mz  and  M3,  so  that 

as  (£*■!)=  !<i(e)  il  112(e)  ^  >i3(£),  M3(e+1)=  Ni(£)  Sr 

Hence 

Ma  (£+1 J  =  N 1  (£)  V  liz  (£)  V  N3  (£)  >  M3  (£+1 ;  =|[i  (£)  V  Na  (£)  V  N3  (£) . 

Also,  we  used  Fig.  1  to  obtain  P,  so  that 

P(£+2) ^  Mi(£+1)^  MbC£+1)  Sa  M3(£+1). 

Substituting  our  formulas  for  Mi.(£+l),  Ma(£+1  j  and  M3(p+1) 

In  the  latter  gives  the  formula  for  P(£+2)  In  the  figure. 
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This  method  of  treating  a  principal  conjunctive  normal  form 
depends  on  that  disjunction  which  has  no  negated  propositions' 
being  one  of  the  factors;  but  it  must  always  be  for  a  positive 
event,  since  otherwise  the  falsity  of  all  the  elementary  pro¬ 
positions  would  make  every  term  of  the  conjunction  true. 

If  the  principal  disjunctive  normal  form  has  n  terms,  the 
principal  conjunctive  nomal  form  has  2^  — n  factors,  and  vice 
versa  (so  the  longer  one  form  is,  the  shorter  the  other).  We 
see  why  this  is  so  in  our  illustration  thus  (omitting  "(p)”  after 
each  "N”  for  brevity). 


Under  the  bar  in  the  second  expression  we  have  the  principal  dis¬ 
junctive  normal  form  of  the  negation  of  the  first  expression,  so 
the  last  expression  (which  is  the  principal  dlsj\mctlve  normal 
form  of  the  original  expression)  is  obtained  by  combining  dis¬ 
junctively  those  5  of  the  8  elementary  conjunctions  which  do  not 
appear  in  the  second  expression. 

When  an  event  can  be  represented  by  a  conjunctive  normal 
form  other  than  the  principal  one,  a  corresponding  simplifica¬ 
tion  can  be  made  in  the  net  constxnictlon  Just  as  in  the  case  of 
disjunctive  normal  forms. 

By  using  a  normal  form  of  either  kind,  we  have  held  the 
lag  £  in  the  representation  to  2. 
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Now  It  may  happen  that  the  most  compact  formula  we  have 
at  hand  to  represent  a  given  definite  positive  event  is  not  a 
normal  form.  Then  we  can  construct  a  nerve  net  of  exactly 
corresponding  structure  and  complexity,  if  we  accept  a  greater 
lag  In  fact,  the  lag  will  be  exactly  the  "depth'^  (or  number 
of  "layers")  in  the  formula  in  terms  of  the  operations  2^  and  V • 
We  shall  see  this  in  Sect.  5*^  (Theorem  2). 

For  some  events,  of  course,  a  lag  of  1  suffices  (or  even 
a  lag  of  0  or  —1  or  —2,  etc.,  if  respectively  the  event  specie 
fles  nothing  about  the  firings  at  time  £  or  times  £—1,  £  or 
times  £-2,  £^1,  £,  etc.  Reduction  of  the  lag  below  2  is  not 
possible  in  general  (with  the  ass\imed  kind  of  neuron).  A 
counterexample  is  the  event  N  j,  (£)  ^  (Na  (iiJ  V  WaCg) )  •  To  repre¬ 
sent  1;h;  t  with  ].ag  1,  the  ne';  would  liao'e  to  eonplei  of  the 

t 

representing  neuron  P  with  endbulbs  belonging  directly  to  Ni, 

Na,  and  N3.  One  readily  sees  that  no  such  net  represents  the 
event  in  question. 

To  hold  the  lag  to  2  in  all  cases  by  use  of  a  normal  form, 
we  may  be  obliged  to  have  a  very  large  number  of  endbulbs 
synapslng  on  a  given  soma,  or  of  axons  emerging  from  a  given 
neuron.  Biologically  there  are  limitations.  A  relatively  small 
increase  in  the  lag  will  cut  these  nvimbers  down.  For  example, 

f: 

a  soma  with  10^  excitatory  endbulbs  sypapsing  on  it  is  replace¬ 
able  with  an  Increase  of  only  2  in  the  lag  by  a  net  made  up  so 
that  only  10®  endbulbs  synapse  on  each  soma  (but,  of  course,  now 
a  large  number  of  neurons  are  necessary) . 
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5.4  Definite  events  In  general; 

2^  —1 

Corollary.  To  each  of  the  2  definite  non— positive 

events-  of  duration  ^  with’  input  neni-^ons  ,  her-  :h:  ^  ner'/?;  3 

having  an  Inner  neuron  which  does  not  fire  at  time  £+2,  if  and 
only  If  the  event  occurs  In  time  ^^+1  to  £. 

Proof.  Denote  the  event  by  E,  and  by  S'  the  complementary 
event  or  negation  of  E,  which  occurs  exactly  If  E  does  not 
occur.  The  set  of  tables,  one  of  which  the  past  must  fit  If  JT 
occurs.  Is  the  complement  (in  the  set  of  all  k  by  ^  tables) 
of  the  set,  one  of  which  the  past  must  fit  If  E  occurs. 

Now  IT  Is  positive,  so  by  the  theorem  (Sect.  5*2),  there 
are  a  net  and  neuron  which  represent  E  by  firing  that  neuron 
at  £+2,  and  therefore  represent  E  by  not  firing  the  neuron  at 
£42. 

Theorem  2.  Consider  any  logical  expression  E  In  tferms  of 
V,  -  '  propositions  Nj^(t)  (l  <  <  ^>  £“^+1  <  i  <  E) 

describing  a  definite  event  E  of  length  ^  with  k  Input  neurons. 
Then  there  Is  a  nerve  net  of  corresponding  structure  which 
represents  E  by  firing  or  by  not  firing,  according  as  E  Is 
positive  or  non—posltlve,  a  certain  neuron  at  time  £+£j  where 
£  Is  the  depth  of  E  In  terms  of  and  V  only. 

proofs  It  will  be  convenient  to  assume  there  are  no  double 
negations  In  E,  as  can  be  arranged  by  use  of  the  law  of  double 
negation  (This  does  not  change  the  depth. ) 

First  we  give  the  treatment  for  the  least  depth  1.  For 
convenience  we  take  ^  to  be  1,  writing  "li,”  "Hz,”  etc.,  for 
"Ei(jB)/’  "N2(li.)."  But  for  ^  >  1  we  would  merely  need  to  use  the 
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method  of  Pig.  3  to  Introduce  neurons  whose  firing  at  jt  =  £ 
represents  the  firing  of  the  various  input  neurons  at  the  earlier 
times. 

By  an  elementary  conjunction  (elementary  disjunction) ,  we 
mean  a  conjunction  of  one  or  more  factors  (terms)  each  of  which 
is  an  elementary  proposition  Nj,  Na,  etc.,  or  a  negated  elemen¬ 
tary  proposition  Fi ,  ^2,  etc.  (By  allowing  one  factor  or  term, 
a  single  proposition  or  negated  proposition  can  be  considered 
as  either  a  conjunction  or  a  disjunction  here.) 

Now  we  have  four  basic  cases  to  treat. 

Case  1;  An  elementary  conjunction  containing  at  least  one 
unnegated  factor,  e.g.,  Ni  2*  Na  N3  ^^4  The  event  is 

then  positivei  so  we  want  to  represent  it  by  the  firing  of  a 
neuron  at  time  £+1.  Use  Fig.  1  to  obtain  this  neuron. 

Case  2:  An  elementary  conjunction  containing  only  negated 
factors,  e.g.,  2-  N2  2<^N3.  The  event  is  non-positive.  But 

now  its  negation  Ni  Na  Ea  positive.  The  j latter  ie  equi¬ 
valent  to  Ni  V  iia  VII3.  Use  the  method  of  Pig.  ^  to  represent 
this  by  a  neuron  fli*lng  at  £+1;  this  neuron  then  represents 
the  original  event  by  non— firing  at  £+1,  as  we  wished  to  have  It 
represented. 

Case  3«  An  elementary  disjunction  containing  at  least  one 
negated  term,  e.g.,  V  ^2  \/ ^3  V  N4  n/N^-  The  event  is  non- 
positive.  But  its  negation  V  IKfa  \/  s/  ^4  s/  is  positive, 
and  the  latter  is  equivalent  to  Ni  ^  Na  Ns  ^  ^4 
Pig.  1  to  represent  the  latter  by  firing  at  £4-1;  then  the  origi¬ 
nal  event  is  represented  by  non— firing  at  £>1. 
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Case  4;  An  elementary  disjunction  containing  only 
unnegated  terms,  e.g.,  NiVNaVNa*  The  event  Is  positive. 

Use  the  method  of  Fig.  2  to  represent  It  by  firing  at  £+1. 

The  cases  are  mutually  exclusive,  except  that  a  single 
unnegated  proposition  N  can  be  considered  as  under  either 
Case  1  or  Case  4,  and  a  single  negated  proposition  ^  as  under 
either  Case  2  or  Case  5.  But  for  one  Input  "only  (which  must 
be  unnegated  for  Fig.  l).  Figs.  1  and  2  coincide;  so  the  treat¬ 
ment  Is  actually  the  same.  (Indeed,  for  or  5^  It  Is  only  to 
have  an  Inner  neuron  which  represents  them  that  any  treatment 
Is  necessary;  otherwise,  we  could  consider  them  as  representing 
themselves  at  time  £. ) 

The  treatment  of  a  formula  with  depth  >  1  requires  only 
Iteration  of  the  processes  used  In  the  four  basic  cases. 

It  will  suffice  to  Illustrate  by  a  complicated  example,  in 
which  the  depth  Is  4: 


4  - 

3  ' - ^ ^ 

2  - :  . - - 

1  - -  - - -  - 

(For  handy  reference  we  took  an  example  with  all  N’s  different, 
but  they  could  be  identified  In  any  combinations.)  The  under¬ 
lines  Indicate  the  parts  of  various  depths.  Also  N3,  F4  and  ^ 
are  parts  of  depth  1,  which  we  could  treat  as  "degenerate" 
elementary  disjunctions,  but  there  Is  no  need  to  consider  them 


thus  here. 


fir 


k  t®  ^ 

,«e  H. 

©Be  »» 


firitiS 
5  t® 

.  <5f  d®?^ 


W  it# 


(l.e.,  Kivi./  -s--'-  ^  the  n®n^ 


ahA  ^3 


tl.. 


4«^ -  E, 

no  longer  all  ar 


^e  show 


'■''®  treat  W  8»“ 


%0 


tii<»»^f*  °'“  g  »n*  tt>e 

It  by  «rt«*  ^  ‘ 


%% 


idh' 


®®® 

ah 


iH  W 


^.%%ersi  hy 
-  ©hhalh 


S.,4I^vp^S^l  • 


I*t' 


th® 


r) 


'fhi® 


L.(EtaWiaCEt2)-  _ 

,  ^  Obtain  a  neuron  «  v*.loH 
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Finally,  consider  the  entire  expression.  Replacing  the 
two  factors  by  their  respective  equivalents,  we  obtain 

Q(£+3)^M3(£+1), 

which  we  treat  by  Case  1  to  obtain  a  neuron  P  which  represents 
it  by  firing  at  p+4. 

Incidentally,  .we  have  discovered  in  the  process  that  the 
event  is  positive — we  did  not  need  to  take  the  trouble  of  set¬ 
tling  which  it  was  at  the  beginning. 

Both  Theorem  1  and  its  corollary  are  corollaries  of  the 
present  theorem,  and  the  nerve  net  constructions  in  Sects.  5*2 
and  50  for  dlsj\mctlve  and  conjiihctlve  normal  forms  are  by 
the  present  method;  so  we  might  have  given  Theorem  2  first. 

Other  logical  operations  which  might  be  used  in  defining 
events' are  definable  in  terms  of  \/ and  ;  e.g.,  (F->G)5F\/G 

is  read  "implies"  or  "if  ...,  then  ..."),  and  (F  — £)  S 
(  (F-^G)  Sc  } . 

Summarizing,  given  any  description,  in  words  capable  of 
being  translated  into  logical  symbolism,  of  a  definite  event, 
we  have  the  means  for  constructing  a  nerve  net  to  represent  it 
of  exactly  corresponding  complexity.  So  the  theory  of  nerve  net 
construction  for  definite  events  is  as  practical  as  one  could 
ask.  The  lag  can  always  be  held  to  2  for  a  given  event,  but 
sometimes  a  greater  lag  will  correspond  to  a  simpler  description 
of  the  event,  and  give  us  a  simpler  net. 

There  may  in  special  cases  be  simpler  nets  than  those  glvai 
by  the  method  of  proof  of  Theorem  2.  We  see  this  by  considering 
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the  condition  for  firing  of  any  inner  neuron  of  thereshold  ^ 
at  time  t,  which  is  that  some  set  of  neurons  having  a  number 
>  h  of  endbulbs  on  it,  and  none  of  the  neurons  having  inhibitoi*y 

4 

endbulbs  on  it,  fire  at  time  ^1.  The  condition  for  not  firing 
is  dual  to  this.  For  example,  the  nets  below  represent  with 
lag  only  1  the  events  described  below  them  using  expressions 
of  greater  depth  (the  upper  by  firing,  the  lower  by  non-firing): 

f  f 


Ni 


vvv 

Na  Na  N4  Nc- 


Na)  V  {Ni  S'Ns)  V  (Na  ^4  ^  I5 

1^1  V  ^2  )  (^1  V  ^3  )  ^  (5?  ^  ^  M.*  ^ 


V  ^ 


(Ni  VN2)&N35.l4 


(El  J I2 )  V  E3  ^  N4 


Figure  15 


(Compare  Fig.  14  with  the  treatment  of  (Ni  vNa)^  Na  £^4  in  the 
long  example  for  Theorem  2.)  We  have  not  undertaken  to  study 
how  much  net  simplification  might  be  gained  by  attempting  to  use 
this  method  systematically  with  the  help  of  appropriate  logical 
transformations . 


5,5  RepresefttAtlon  of  events  in  general :  We  can  now  pro^e 
the  remark  we  made  in  Sect.  1  that  there  is  no  loss  of  generality 
in  considering  the  representation  of  an  event  to  consist  df  the 
firing  or  the  non— firing  (as  appropriate)  of  a  single  neuron. 
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By  saying  that  an  .  ‘^'cunning  a  '.inn 

with  the  moment  p)  Is  represented  in  a  nerve  net  at  a  certain 
time  £+•£  (s  >0),  we  mean  that  some  property  of  the  state  of 
the  net  at  time  2+£  Is  equivalent  to  the  event  having  occurred 
ending  at  time  £;  i.e.,  according  as  the  event  did  or  did  not 
occur,  the  net  will  or  will  not  have  that  property. 

But  what  happened  at  times  <  £  can  only  affect  the  state 
of  the  net  at  times  £f£  for  £  >  0  via  the  state  of  the  net  at 
time  £. 

Say  besides  the  k  input  neurons  there  are  m  Inner  neurons. 

The  state  of  the  npt  at  time  £  consists  of  the  condition 

(firing  or  non— firing)  of  each  of  the  m+k  neurons.  Thus,  there 
nn-k 

are  exactly  2—  —  possible  states  at  time  £.  There  are  2® 
properties  of  the  state  of  the  net  at  time  £.  Any  event  ending 
at  time  £  which  can  be  represented  at  time  £+s  Is  thus  equivalent 
to  one  of  these  2^  properties  of  the  state  at  time  £. 

But  for  each  of  these,  by  applying  the  method  of  proof  of 
Theorem  1  or  Its  corollary,  or  of  Theorem  2,  to  all  the  m+k 
neurons  (instead  of  only  the  Input  ones)  and  to  only  the  moment 
£  (instead  of  the  Interval  £—^+1  to  £) ,  we  can  add  additional 
neurons  to  get  a  neuron  P  which  will  fire  or  not  fire  (according 
as  the  property  of  the  m+jo  neurons  at  time  £  Is  positive  or  not) 
at  time  £+2,  If  and  only  if  the  mfk  neurons  fulfill  the  property 
at  time  £;  and  hence.  If  and  only  If  the  event  in  question 
(referring  to  input  neurons  and  ending  at  time  £)  occurred. 

Incidentally,  we  have  not  made  any  assumption  here  whether 
the  event  in  question  Is  definite  or  not.  ■ 
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In  organizing  a  complex  of  stimuli  Into  a  complex  of 
responses  (Sect,  l)  as  economically  as  possible.  It  is  to  be 
expected  that  the  representation  of  events  will  not  always  be 
compressed  Into  the  form  of  the  firing  or  non— firing  of  a  single 
neuron . 

5.6  Nerve  nets  without  circles;  A  circle  (of  length  £) 
in  a  nerve  net  Is  a  set  of  distinct  neurons  such  that 

has  an  endbulb  on  (i,  l)  and  has  one  on  Ni. 

Theorem  3:  Given  any  nerve  net  without  circles  and  any 
Inner  neuron  N  In  that  net,  the  firing  (non— firing)  of  that 
neuron  at  time  £+1  Is  equivalent  to  the  occurrence  of  a  definite 
positive  (non-posltlve)  event  ending  at  time  £. 

This  theorem  is  stated  for  positive  events  by  McCulloch 
and  Pitts  (1943). 

Proof.  Whether  N  fires  at  £+1  Is  completely  determined  by 

I  I 

the  firing  or  non— firing  at  £  of  those  neurons  having 

I  I  “* 

endbulbs  on  N.  Consider  those  of  Ni,...,N^  which  are  inner 
neurons,  and  repeat  the  argument.  Since  there  are  no  circles, 
any  chain  of  neurons  each  impinged  upon  by  an  endbulb  of  the 
preceding  must  terminate.  Let  ^+1=  the  length  of  any  longest 
such  chain;  a  longest  must  exist  since  there  are  finitely  many 
such  chains,  and  ^  >  1  since  N  is  inner.  Then  the  process  ter¬ 
minates  after  ^  steps.  Thus,  the  firing  or  non— firing  of  N  at 
£+1  Is  completely  determined  by  the  firing  or  non— firing  of 
certain  input  neurons  at  times  £-^+1  to  £;  l.e.,  it  is  equiva¬ 
lent  to  a  definite  event  of  duration  The  event  must  b® 
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positive,  as  firing  can  only  be  propagated  but  not  originated 
under  the  law  for  an  inner  neuron's  firing  (Sect.  . 

Then  of  course  N's  non— firing  at  time  g+1  is  equivalent  to 
the  complementary  event,  which  is  non— positive. 

(Any  definite  event  is  expressible  by  a  logical  formula, 
hy  a  principal 'disjxinctlve  normal  form  as  in  Sect.  5*2. 

So  a  priori  there  is  a  formula.  By  utilizing  the  condition 
for  firing  at  each  synapse,  which  we  formulated  in  words  in 
the  last  paragraph  of  Sect.  5-^  and  could  have  in  symbols,  one 
can,  of  course,  build  up  a  formula  in  ^  stages,  as  McCulloch— 
Pitts  indicate.) 

Corollary;  For  a  net  without  circles,  any  event  ending 
at  time  £  which  can  be  represented  by  the  firing  (non— firing) 
of  a  given  inner  neuron  M  at  a  certain  time  £+£  (^  >  2)  is 
definite  and  positive  (definite  and  non-positive). 

Proof ;  For  by  the  theorem,  the  condition'  for  the  firing 
of  N  at  time  £+£  is  the  occurrence  of  a  definite  positive  event 
ending  at  time  £+S;— 1.  But  since  by  hopothesls,  N's  firing 
represents  an  event  ending  at  time  £,  the  input  over  time 
£+1 , . . .  ,£+8;-l  cannot  affect  whether  the  aforesaid  definite  posi¬ 
tive  event  occurs  ending  at  time  £+jS— 1.,  So  in  fact  that  definite 
positive  event  can  be  taken  to  refer  only  to  a  time  ending  at  £. 

This  corollary  constitutes  the  converse  of  Theorem  1  and 
corollary  (or  Theorem  2).  Likewise,  any  event  ending  at  £ 
represented  by  a  state,  or  a  property  of  the  state,  at  a  time 
£+£  of  a  net  (s^o)  without  circles,  is  definite. 
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6.  Indefinite  Events — Preliminaries; 

6.1  Some  examples;  Consider  the  following  nerve  net 
(with  a  circle  df.  length  1  consisting  of  M). 

M 

Figure  15 

If  at  some  time  t  £  the  neuron  N  flresi  then  the  firing 
of  M  at  time  £+1  (and  at  every  subsequent  time)  will  follow. 

In  ssnnbols, 

<  ^(e)- — >M(£+1) 

("(^).  .  "  is  read  "there  exists  a  jt  ^  £")•  do  not 

have  equivalence  ("=")  Instead  of  merely  Implication  (" — 3f 
past  time  Is  taken  as  Infinite,  since  the  firing  of  M  at  time 
£+1  can  also  be  explained  by  firing  of  M  at  every  past  moment, 
without  N  having  ever  fired. 

Similarly,  the  net 

V 

Figure  l6 

will  only  fire  at  time  £4-1  If  N  has  fired  at  all  past  times; 
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in  symbols 

(‘)t  <  -f —  ”(e+i) 

(read  "(i)^  ^  ”  as  "for  all  ^  ^  but  not  conversely,  for 

M  may  fall  to  fire  at  time  £+-1  when  (t)^  ,  N{£)  Is  true,  by 

falling  to  fire  over  all  past  time. 

(M)t  and  (t)^  ^  ^(£)  are  simple  exan^les  of 

events  not  deferring  to  a  definite  period  of  past  time;  and  we 
see  that,  under  the  assumption  that  past  time  is  infinite,  the 
nets  shown  do  not  represent  them,  by  firing  at  time  £fl,  in  the 
sense  of  equivalence  (the  first  is  represented  in  the  sense  of 
"necessity”  only,  the  second  "sufficiency"  only). 

If  we  attempt  to  represent  the  former  by  non— firing,  we 
have  a  net 


for  which  (Et)^  ^  ^  aot  conversely. 

The  difficulties  encountered  in  these  three  examples  are 
not  escapable  by  using  other  nets  to  represent  the  events,  or 
in  other  examples  of  indefinite  events,  but  csnstlttite  the 
general  rule  for  indefinite  events.  We  shall  show  in  Appendix  1 
(Theorem  7,  to  be  read  after  Part  II)  that,  under  the  asstimptlon 
of  an  infinite  past,  an  event  can  be  represented  (in  the  sense 
of  equivalence)  by  the  firing  or  by  the  non— firing  of  a  certain 
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Inner  neuron  at  time  £+£  (for  any  fixed  £  ^  l)>  only  if  the 
event  is  definite.  For  any  net  and  inner  neuron  and  and 
any  indefinite  event,  it  must  either  be  possible  to  have  a 
past  for  which  the  event  does  not  occur  and  the  neuron  fires, 
or  one  for  which  the  event  does  occur  and  the  neuron  does 
not  fire,  or  both. 

Of  course,  any  living  organism  or  actually  constructed 
robot  has  only  a  finite  past.  The  mentioned  result  shows  that 
now  we  must  take  this  into  account;  othenrtnl«»:  we  might  have 
been  tempted  to  use  the  fiction  of  an  infinite  past  to  simplify  . 
the  theory. 

6.2  Initiation;  Accordingly,  let  us  assume  that  the  past 
for  our  nerve  nets  goes  back  from  £  (the  present)  a  certain 
finite  time  only,  the  first  moment  of  which  shall  be  1  on  our 
time  scale.  (We  find  it  more  convenient  notationally  to  call 
the  first  moment  ^  -  1  than  ^  =  0,  but  if  we  think  of  each 
positive  Integer  ^  as  referring  to  the  final  Instant  of  a  unit 
interval,  this  does  make  time  start  from  0.) 

In  seeking  to  represent  events,  we  shall  now  assume  the 
right  not  only  to  construct  the  nerve  nets  as  we  please,  but 
also  to  fix  the  state  (firing  or  non— firing)  of  each  inner  neuron 
at  time  1.  That  is,  we  study  representation  of  events  in  nerve 
nets  started  with  a  given  internal  state  at  the  initial  moment  1. 

The  range  of  the  time  variables  in  our  logical  formulas 
shall  now  be  the  Integers  from  1  forward,  and  this  shall  be  the 
only  part  of  the  past  we  talk  about  except  when  we  make  it  plain 
we  Intend  otherwise. 
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Now  the  nerve  net  of  Fig.  15,  started  at  t  =  1  with  M 
quiet,  represents  by  the  firing  of  M  at  £+1  the  event 
(Et).  -  Pig-  l6,  started  with  M  firing, 

represents  likewise  That  of  Fig.  17  started 

with  M  firing  represents  (Et),  .  ..Ji(t)  by  the  non— firing  of  M 

JS  £ 

at  £+1.  Thus,  the  two  nets  of  Fig.  15  and  17  are  able  to 
remember  If  N  has  fired  since  their  beginning  by  changing  M  from 
the  state  It  had  Initially;  while  the  net  of  Fig.  16  Is  able  to 
recognize  that  N  has  never  failed  to  fire  by  preserving  M  In 
the  state  It  had  originally,  as  Householder  and  Landahl  (19^5) 
have  commented  (p.  109). 

To  represent  (t).  .  ^(t)  either  by  the  firing  or  the 

~  _  S  J&”  ” 

non— firing  of  a  neuron  In  a  net  with  only  N  as  Input  neuron, 
at  least  one  Inner  neuron  must  be  fired  Initially.  For  were 
all  Inner  neurons  quiet  at  time  t  ^  2,  then  In  case  N(l) 

(l.e..  If  the  Input  neuron  N  does  not  fire  at  time  ^  »  1),  all 
neurons  would  be  quiet  at  ^  =  2.  So  the  state  of  the  Inner 
neurons  at  _t  f>  2  would  then  be  Indistinguishable  from  that  at 
t  =  1.  Hence,  the  net  at  any  time  £+1  ^  3  would  have  the  same 
state  whether  the  past  Is 

t  1  2  3  4  ... 

N( t)  0111..., 

which  makes  ^ 


t  12  3  4 

N(t)  1111 
which  makes  (;t)^  ,  time. 
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The  case  In  which  all  inner  neurons  are  initially  quiet  . 
is  natural  neurologically;  the  other  case  leaves  it  unexplained 
how  the  firing  of  certain  inner  neurons  is  to  be  produced  at 
t  =  1.  Of  course,  a  natural  explanation  would  be  available 
b,y  setting  the  time  origin  back,  if  the  initial  state  in  ques¬ 
tion  is  one  which  could  be  brought  about  by  a  suitable  pattern 
of  firings  of  the  input  neurons  over  a  finite  preceding  time 
at  the  beginning  of  which  all  the  inner  neurons  are  quiet.  But 
this  is  not  so;  e.g.,  in  the  case  of  the  simple  net  for 
(i)t  <  (Pig.  16). 

Although  (t).  .  cannot  be  expressed  (for  N  as  sole 

^  2r“ 

input  neuron)  without  having  an  initially  fired  inner  neuron, 
(lii)u  <  ^  ^-^s  <  ^  ^-^u  ^  t  ^  Input 

neurons  K  and  N  can  be>  by  the  following  net,  in  which  P  fibres 
at  g+2  if  and  only  if  the  event  occurs. 


The  neurons  K,  Li  and  Lg  act  as  a  starting  circuit,  which  can 
only  be  used  once,  for  the  generality  circuit  N,  M  and  P.; 

The  device  with  a  modification  is  gehelral.  Suppose  we  have 
given  a  net  in  which  an  event  is  represented  by  the  firing  of  a 
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certain  inner  neuron  at  time  £+£  if  certain  inner  neurons  are 
initially  fired. 

First,  let  us  add  a  starting  circuit  K,  Li  and  La  from  a 
new  Input  neuron  K,  with  axons  from  its  neuron  leading  to  alL 
the  same  neurons  and  with  the  same  kinds  of  respective  endbulbs 
as  the  axons  from  each  of  the  inner  neurons  which  were  initially 
fired  in  the  given  net. 

Furthermore,  each  input  neuron  N  of  the  original  net  we 
now  make  an  inner  neuron  N*  with  a  threshold  of  2,  and  we 
insert  new  neurons  N  and  R,  the  former  taking  over  the  role  of 
the  original  N  as  input  neuron,  and  the  latter  an  inner  neuron 
as  shown.  The  heavy  line  stands  for  the  axons  which  lead  from 
the  original  N. 


This  accomplishes  the  double  purpose  of  lagging  the  input  from 
N  by  1  and  of  blocking  it  for  moments  prior  to  the  first  moment  u 
at  which  K  is  fired. 


Now  with  all  inner  neurons  initially  quiet,  no  neuron 
except  N  can  fire  until  the  first  moment  u  at  which  K  fires. 
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Then  at  moment  u+1  the  neuron  Li  takes  over  the  role  of  the 

I 

original  initially  fired  inner  neurons,  while  N  carries  the 
input  of  N  at  u.  At  every  subsequent  moment  u+v+l  (v  >  l), 
all  of  the  original  neurons  (covinting  N  as  the  original  N) 
will  behave  as  they  fonnerly  would  have  at  time  v+1,  if  the 
present  input  over  u,  ...,u4-v  had  been  the  input  over  time 
1 ,  •  •  •  ,  V"4"  1  • 

So  the  output  neuron  will  fire  at  g^s^l,  if  and  only  if 
the  event  now  occurs  relative  to  jb  =  u  instead  of  to  ^  =  1; 
i.e.,  we  have  a  representation  of  the  event  redefined  to  r*fer 
not  to  the  whole  past  but  to  the  past  beginning  with  t  -  u, 
and  with  an  Increase  of  1  in  the  lag  in  the  representation. 

Now  if  it  is  assxrmed  that  there  are  conditions  in  the 
environment  which  would  continually  stimulate  K  to  fire,  or 
that  4t  least  such  a  condition  exists  at  ^  ■  1,  then  our  net 
will  represent  the  event  relative  again  to  the  whole  past 
(since  now  u  =  l).  Thus,  we  are  provided  with  a  "natural"  way 
of  getting  a  representation  of  any  event,  referring  to  the  input 
neurons  besides  K,  which  could  be  represented  "unnaturally"  by 
the  firing  of  a  neuron  in  a  net  started  with  some  initially 
fired  input  neurons. 

Here  "natural"  means  only  that  we  do  not  need  to  go  out¬ 
side  the  McCulloch— Pitts  laws  of  neural  behavior  to  fire  some 
inner  neurons  at  t  «=  1;  but  the  starting  circuit  K,  Li,  La,  and 

I 

the  blocking  circuit  K,  R,  N  are  not  thought  of  as  plausible 
mechanisms  biologically.  However,  our  first  aim  is  to  see  what 
is  at  all  possible,  and  one  can  then  seek  other  and  perhaps  more 
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natural  ways  for  accomplishing  the  same. 

^hls  argument  that  by  assuming  an  initially  stimulated 
input  neuron  K  we  can  avoid  having  to  have  any  initially  fired 
inner  neurons  applies  only  to  representation  of  events  by 
firing  a  neuron,  if  for  representation  by  non— firing  at  £[♦■£+1 
one  wishes  that  the  output  neuron  fire  at  times  1,...,£+1. 

The  (juestlon  involved  will  be  analyzed  in  Sect.  6.3. 

The  same  construction  but  omitting  the  K  and  Lg  of  the 
starting  circuit  and  the  delaying— blocking  neurons  R  and  N*, 
and  firing  Li  initially,  shows  that  it  is  always  possible,  if 
we  are  to  use  Initially  stimulated  inner  neurons,  to  hold  the 
number  of  them  to  1,  without  any  increase  in  the  lag.  This 
again  is  for  the  case  of  representation  by  .firing.  For  repre¬ 
sentation  by  non— firing,  the  situation  is  slightly  more  compli¬ 
cated,  and  we  shall  not  go  into  it. 

As  stated,  we  ordinarily  consider  nets  only  for  a  speci¬ 
fied  initial  state  of  the  inner  neurons.  However,  McCulloch 
and  Pitts  consider  the  problem  of  "solving"  nets  with  their 
Initial  condition  unspecified.  To  "solve"  for  a  given  inner 
neuron  P,  say  at  time  li+l,  means  then  to  find  for  what  Inputs 
over  time  !,...,£,  and  what  initial  states  of  the  inner  neurons,. 

P  will  fire  at  Jb  -  £fl.  Now  in  the  following  net,  the  necessary 
and  sufficient  condition  that  P  fire  at  £+1  is  that  N  fire  at 

all  times  <  £  and  both  Mj  and  Ma  fire  at  time  t  «  1. 

♦ 

P(£+l)= 

(t)t<^(t)^M,{l)£Ma(l) 

Figure  20 
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This  seeifis  to  be  coimterexample  to  the  formula  next  after 
(9)  on  p.  126  of  McCulloch— Pitts  (19^3) >  the  proof  of  which  we 
did  not  follow;  for  if  we  understand  the  formula  correctly,  it 
implies  that  the  condition  for  firing  should  only  require  the 
existence  of  one  (suitably  chosen)  neuron  known  to  fire  initially. 
In  this  example  we  cannot  conclude  that  P  flr'es  at  g+l  on  the 
basis  of  any  Information  which  tells  us  only  that  one  of  the 
neurons  fires  at  ^ 1.  (Our  1  seems  to  be  their  0.)  This 
apparent  counterexample  has  discouraged  us  fi^m  further  attempts 
to  decipher  Part  III  of  the  McCulloch— Pitts  1943* 

6.3  Definite  events  reconsidered;  Now  that  we  have 
Introduced  the  assvimptlon  that  the  past  for  a  nerve  net  is 
finite,  we  must  reexamine  the  treatment  of  definite  events  which 
was  given  in  Sect.  5* 

What  happens  now  when  £  <  l.e.,  when  the  period  of  time 

to  which  the.  event  is  supposed  to  refer  extends  back  to  before 
the  moment  _t  *  1? 

Generally,  one  may  suppose  that  the  durations  ^  of  defi¬ 
nite  events  which  are  significant  for  aui  organism  will  be  small 
in  relation  to  the  age  £  of  the  organism  at  which  the  event  is 
significant. 

This,  however,  does  not  enable  us  to  dismiss  the  problem. 

For  to  make  the  theory  of  nerve  net  control  accurate,  we  should, 
for  each  definite  event  considered,  either  (a)  show  that  an 
"hallucination"  that  the  event  has  occurred  arising  during  the 
first  l+£  moments  of  life  could  not  have  any  serious  effect 
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on  behavior,  or  (b)  provide  against  the  occurrence  of  such  an 
hallucination  in  the  first  l+£  moments. 

When  we  use  definite  events  to  build  indefinite  ones, 
e.g.,  the  event  consisting  df  a  certain  definite  event  having 
occurred  ending  at  some  time  £  ^  £  (briefly,  the  memory  of  the 
definite  event  having  occurred),  such  an  hallucination  could 
conceivably  have  a  long-term  effect,  even  if  it  has  no  Immediate 
effect  on  behavior. 

The  solution  by  (a)  is,  of  course,  outside  the  present 
investigation,  and  belongs  rather  to  the  full  problem  of  orga¬ 
nizing  stimuli  into  responses  (Sect.  1). 

For  organisms,  the  picture  of  the  nervous  system  coming 
into  activity  in  toto  at  a  fixed  moment  Jb  ^  1  is  Implausible. 

But  this  means  only  that  organisms  (at  least  those  vrtilch  sur¬ 
vive)  do  solve  the  problem  for  their  process  of  coming  into 
activity. 

For  machines,  it  is  familiar  that  starting  difficulties 
may  have  to  be  taken  into  account  by  the  engineer. 

To  take  a  fictitious  Illustration,  consider  the  case  of 
the  "rat  satellite  robots"  for  the  Tuvian  Navy.  A  rat  satellite 
robot  is  Intended  to  go  about  a  ship,  and  vrtienever  after  three 
hours  (=  ^—1  moments)  it  has  not  smelled  a  rat,  and  at  the  next 
moment  (the  th)  land  is  in  sight,  the  robot  abandons  ship. 

The  robots  were  ordered  from  RAND  and  were  built  by  the 
Robotry  Section  from  blueprints  prepared  by  the  Logicians  Grk>up 
on  the  basis  Of  the  theory  in  Sect.  5  above,  with  two  input 
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neurons,  Ni  which  Is  fired  by  the  smell  of  a  rat,  and  N2, which 
Is  fired  by  the  sight  of  land.  The  Inner  neuron  P,  which  fires 
at  time  ^2  If  the  event  "no  rat  smelled  for  ^—1  moments,  land 
seen  at  the  th  moment”  has  occurred  during  the  time  £—^+1  to  £, 
was  connected  to  an  effector  mechanism  for  abandoning  ship  at 
time  £+5 . 

Suitable  ceremonies  were  scheduled  for  the  occasion  of 
their  Installation  In  the  harbor  of  the  Tuvian  Naval  Base. 

When  the  occasion  arrived,  they  were  placed  on  board  the 
ships,  and  their  batteries  were  connected  up  supplying  power 
for  operating  the  nerve  nets  and  effector  mechajilsms.  But  three 
moments  later.  Just  as  the  Tuvian  Grand  Admiral  was  congratula¬ 
ting  the  RAND  delegate,  all  the  robots  went  overboa3?d! 

Proceeding  to  details.  It  Is,  of  course,  a  matter  of 
definition  how  we  shall  Interpret  "events  of  duration  when 
p  <  But  whatever  definition  Is  adopted,  we  must  keep  the 
facts  about  nerve  net  behavior  straight. 

We  shall  (as  best  suits  our  present  purpose,  which  Is  to 
lay  a  firm  basis  for  the  theory  In  Sect.  7)  say  that  an  event  E 
of  duration  ^  can  only  have  occured  ending  at  £  vriaen  £  >  ^. 

Then,  of  course,  the  logical  formulas  we  have  used  to 
represent  the  events  In  Sect.  5  are  not  ccmiplete.  If  Ei  Is  the 
formula  which  described  a  definite  event  of  length  ^  there, 
the  formula  S  which  describes  It  fully  now  Is  Ei^£  >  ^.  The 
negation  E  of  this  Is  5iV£  <  ^  >  while  the  formula  for  the 
"complementary"  event  of  duration  ^  Is  rather  5i<^£  >  Thus, 
some  care  Is  now  necessary  In  connection  with  the  operation  of 
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negation.  The  theory  In  Sect.  5  Is  applicable  to  the  part  of 
the  formula  which  does  not  give  the  time  reference;  l.e.,  the 
relationships  studied  there  apply  to  the  Et  and  Fi. 

Now  consider  a  nerve  net  as  constructed  for  Theorem  1  In 
Sect.  5*2  to  represent  an  event  occupying  the  time  Interval 
2-^+1  to  £  of  length  ^  by  firing  a  certain  Inner  neuron  (the 
"output  neuron")  at  time  £+2. 

Using  this  net  under  the  restriction  now  that  the  life  of 
the  net  starts  with  a  certain  moment  ^  1,  and  under  the  sti¬ 

pulation  that  at  that  moment  all  Inner  neurons  are  qvilet.  It  Is 
clear  that  the  output  neuron  will  fire  at  any  time  £+2  for 
£  >  ^  correctly;  l.e..  If  and  only  If  the  event  occurrrd  In  the 
time  £-^+1  to  £. 

But  the  net  might  also  fire  at  a  time  <  ^+2;  namely,  this 
could  happen  if  and  only  If  the  event  Is  such  that  our  present 
Initial  condition  of  the  Inner  neurons  (all  quiet)  Is  one  which 
could  also  take  place  In  Sect.  5*2  at  some  moment  m  vrtiere  £--^+2 
<  m  <  £fl  for  some  occurrence  of  the  event  In  £-2[+l  £• 

So  assume  (in  the  context  of  Sect.  5*2)  that  we  hawe  an 
oecuri*ence  of  the  event  In  the  course  of  which  all  Inner  neurons 
are  quiet  at  ^  =  m. 

The  state  of  the  Inner  neurons  at  time  m  would  then  have  to 
be  the  same  (l.e.,  all  quiet)  If  the  'table  describing  the 
past  Is  altered  to  show  only  0*s  for  all  Input  neurons  at  all 
times  t  <  m.  For  from  a  past  consisting  entirely  of  non— firings 
prior  to  m,  no  firing  of  any  Inner  neuron  can  be  produced  at 


time  m. 
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Since  the  state  of  the  inner  neurons  is  unchanged  at 
time  m,  and  the  Inputs  for  jb  =  m, . » .  are  unchanged,  the 
output  neuron  still  fires  at  ^  =  £+2.  So  by  Theorem  1,  the 
event  still  does  occur  in  time  £-^+1  to  £;  l.e.,  we  now  have  an 
instance  of  the  event  occurring  in  which  no  input  neurons  were 
fired  in  £-j^+l  to  m-1;  in  particular  (since  m  >  £-^+2),  the 
event  can  occur  with  input  0  on  all  its  input  neurons  at  its 
first  moment 

Conversely,  if  this  is  the  case,  the  ontput  neuron  fire 

now  at  time  ^+1,  if  in  times  1,...,^— 1  the  Inputs  are  what  they 
could  be  in  Sect.  5*2  for  the  moments  E-^+2  to  £  of  an  occurrence 
of  the  event  with  only  O's  for  £-^+1. 

Call  a  definite  event  of  length  ^  prepositive  if  the  event 
can  occur  only  when  some  input  neuron  fires  in  its  first  moment 
£-i+l;  l.e.,  the  selections  from  among  the  2®  possible  k  x  ^ 
tables  which  describe  occurrences  of  the  event  all  have  at  least 
one  1  in  their  bottom  row.  The  prepositive  events  are  a  subclass 
of  the  positive  events. 

Now  we  have  shown  that  a  necessary  and  sufficient  condi¬ 
tion  that  no  ”halluclnatlon''  be  possible  (in  the  sense  of  the 
output  neuron’s  firing  at  a  time  t  when  the  event  has  not  occurred 
ending  at  time  tp5)  is  that  the  event  be  prepositive. 

We  gave  the  reasoning  for  the  case  in  which  the  event  is 
to  be  represented  by  firing  of  a  neuron  at  time  £+2  (corres¬ 
ponding  to  Theorem  l),  but  it  applies  equally  well  to  the  other 
cases  in  Sect.  5;  i.e.,  to  representation  by  firing  at  £+i8  for 
any  given  £  >  1,  or  to  representation  by  non— firing  at  £+£ 
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(then  hallucinations  are  always  possible,  since  the  event  Is 
necessarily  non— positive  by  Theorem  3  or  its  corollary) ,  or 
to  representations  by  a  property  of  the  state  at  a  certain 
time  p+£  (s  >  0).  (In  giving  the  sufficiency  proof,  we  write 
the  inequality  on  m  now  £—^+2  m  <  £+£,  and  change  the  input 
for  t  ^  max(in— 1,£) . ) 

Most  but  not  all  events  we  may  wish  to  consider  will  be 
prepositive. 

The  analysis  is  valid  for  any  net  which  operates  correctly 
when  £  >  ^,  whether  constructed  as  in  Sect.  5  or  not,  and 
started  now  with  all  inner  neurons  quiet. 

If  a  positive  event  of  length  ^  is  not  prepositive,  we  can 
build  a  net  which  represents  it  by  firing  a  neuron  P  at  time 
£+2  (or  £+£  for  some  £  >  1),  if  this  net  is  started  at  ^  «  1 
with  one  of  its  inner  neurons  fired  (but  all  others  quiet),  as 
follows.  We  simply  take  an  inner  neuron  L  as  in  Figure  21, 
Initially  fired  (as  the  '*+"  indicates). 

# 

Figure  21 

We  then  treat  this  as  though  it  were  an  additional  input  neuron, 
required  to  fire  at  t  «  £-2!+l»  applying  the  method  of  net 
construction  of  Sect.  5*  (Of  course,  then  more  than  one  axon 
may  be  required  from  L  to  other  neurons.) 
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This  device  also  makes  It  possible  to  represent  non— 
positive  events  of  length  ^  by  firing  of  a  neuron  at  time  £+2 
(or  £+£  for  some  £  >  in  Sect.  5  they  would  have  to  be 
represented  by  non— firing. 

Another  device  for  fixing  a  net,  constructed  as  In  Sect.  5 
to  represent  a  positive  but  not  prepositive  event  (then  ^  >  2), 


SO  that  no  hallucination  can  be  produced.  Is  to  let  the  Inhibitory 


of  Ijx  f'l  \  fc 

of  P  of  the  net  of  Sect 


bcwfoy;  -lev  ;’mplngv-  up  0:0  -.hH  jafpot  n*u 

5.  The  number  of  the  L’s  is  ^+£-2- 


can  also  use  'fnls  to  fix  nets  <;on3t-ructed  as  in  Sect,  5 
to  represent  a  non-positive  event  by  non— firing  at  £+-£,  11  w* 
change  the  endbulb  of  Li  which  is  to  impinge  on  P  to  a  set  of 
excitatory  endbulbs  equal  In  nxiraber  to  the  threshold  of  P,  and 
also  fire  P  Itself  at  time  1.  (If  »  2,  no  L's  are  added.) 

The  devices  of  Figs.  21  and  22  seem  artificial,  and  not 
likely  to  be  found  in  organisms.  We  point  them  out  to  save  the 
need  for  making  an  exception  of  non— prepositive  events  In  the 
theory.  If  mechanical  realizations  of  McCulloch— Pitts  neurons 
are  used  in  contirolllng  robots,  such  devices  might  be  useful. 
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The  upshot  of  the  analysis  is  that  only  by  reference  to 
artificially  produced  firing  in  inner  neurons  at  Jt  =  1  could 
an  organism  recognize  complete  absence  of  stimulation  of  a 
given  duration,  not  preceded  by  stimulation;  it  would  not  know 
whether  the  stimulation  had  been  absent,  or  whether  it  had  itself 
meanwhile  come  into  existence. 

If  instead  of  the  initially  fired  inner  neuron  of  Fig.  21 
we  use  an  input  neuron  subject  to  continual  environmental  stimu¬ 
lation,  then  all  events  can  be  taken  to  be  prepositive  by 
referring  them  to  the  class  of  input  neurons  as  enlarged  to 
Include  K. 

This  is  plausible  biologically,  if  we  also  grant  that 
the  mathematical  model  is  probably  too  exact  in  that  it  gives 
too  much  emphasis  to  a  single  neuron  at  a  single  moment  of 
time  (.0005  sec.).  It  is  unlikely  that  any  such  input  at  a 
single  moment  would  by  itself  result  in  any  significant  overt 
action  or  memory. 

Having  chosen  to  Investigate  a  precise  model,  it  is  not 
to  be  expected  that  all  aspects  of  this  model  will  be  equally 
pertinent  to  the  reality  from  which  the  model  is  abstracted. 

6.4  Why  consider  indefinite  events?  Since  the  lifetime 
of  an  organism  or  machine  is  always  finite,  having  an  end  as 
well  as  a  beginning,  why  is  it  not  sufficient  to  consider  only 
definite  events? 

The  number  of  moments  (identifying  a  moment  with  a  synaptic 
delay  of  .0005  sec.)  in  a  human  lifetime  of  100  years  is  of  the 
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order  of  5  x  10^^. 

To  construct  a  nerve  net,  treating  events  as  definite, 

that  would  account  for  behavior  at  60  years  of  age  Influenced 

by  stimuli  at  10  years,  we  would  need  chains  of  neurons  of 
12 

length  1.5  X  10  .  If  the  event  were  at  all  complicated,  we 

would  need  large  numbers  of  such  chains.  Moreover,  we  would 
need  further  mechanism  to  provide  for  this  same  behavior  occur¬ 
ring  at  6l  years  or  59  years  due  to  stimuli  at  10  years,  or 
Indeed  for  each  value  of  d  where  d  ranges  from  the  smallest 
elapsed  time  after  10  years  at  which  the  behavior  can  be 
Influenced  up  to  the  greatest,  and  Is  measured  In  units  of 
.0005  sec.  We  do  not  necessarily  need  a  whole  new  set  for 
each  value  of  d,  since  many  neurons  can  be  made  to  serve  In 
common  for  various  values  of  d,  e.g.,  the  delay  chains  for 
various  values  of  d  greater  than  a  given  one  di  could  have  their 
first  di  neurons  in  common.  But  at  least  each  intermediate 
value  of  d  would,  up  to  the  greatest  in  question,  require  some 
structural  additions,  new  axons  if  not  new  neurons. 

All  this  would  have  to  be  duplicated  for  every  sort  of 
event  which  occurring  at  one  time  could  influence  behavior  at 
all  later  times  In  life. 

1  fi 

The  total  number  of  neurons  is  only  of  the  order  of  lO”^  . 

To  use  definite  events  as  a  mathematical  basis  for  explain¬ 
ing  human  behavior  in  all  its  flexibility  over  a  lifetime  of 
5  X  10^®  moments  thus  appears  altogether  unrealistic. 

To  emphasize  what  is  meant,  take  the  case  of  Solomon 
Grundy.  On  the  afternoon  of  Monday  he  burns  his  hand  on  the 
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stove.  Then  one  nerve  net  tells  him  not  to  touch  the  stove 
on  Tuesday,  a  different  one  (at  least  in  part)  on  Wednesday, 
and  so  on. 

If  he  outlives  the  life  expectancy  for  which  his  delay 
chains  are  designed,  he  must  thereafter  suffer  an  advancing 
amnesia;  for  each  day  added  beyond  his  expectancy  at  the  end  of 
life  he  completely  forgets  one  day  at  the  beginning. 

Humans  and  animals  do  not  function  in  this  way,  though 
simple  mechanisms  for  learning  and  subsequent  forgetting  in 
robots  could  be  devised  on  this  basis. 

Indeed,  calculations  on  the  amount  of  information  recorded 
in  the  memory  (cf.  McCulloch  1949)  make  it  difficult  to  explain 
memory  entirely  in  terms  of  McCulloch— Pitts  neurons  on  any 
basis,  a  fortiori,  certainly  not  in  such  an  uneconomical  way  as 
by  setting  up  only  nets  for  definite  events.  So  it  is  neces¬ 
sary  (if  perhaps  in  the  end  it  will  not  be  sufficient)  to  go 
beyond  the  present  stage  of  our  analysis. 

It  thus  appears  that  the  appropriate  mathematical  abstrac¬ 
tion  for  us  now  is  to  treat  the  problem  of  explaining  behavior 
as  though  organisms  and  machines  were  immortal,  having  an 
infinite  future  though  a  finite  past.  We  want  to  provide  for 
behavior  that  could  be  used  ^  infinitum,  if  merely  the  nerve 
net  and  effector  mechanisms  were  immortal. 

By  trying  to  provide  for  behavior  over  an  infinity  of 
time  by  a  finite  mechanism,  we  have  a  model  for  the  real  prob¬ 
lem  of  providing  for  complex  behavior  over  a  long  finite 
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lifetime  by  a  relatively  small  mechanism. 

The  questions  of  reducibllity  of  other  mechanisms  to 
McCulloch— Pitts  nerve  nets  (not  always  without  increasing  the 
size  of  the  mechanisms)  is  significant  on  this  basis,  but 
trivial  on  the  basis  of  explaining  behavior  over  a  fixed  fintt« 
time  only. 

7.  Regular  Events: 

7.1  "Regular  events”  defined;  We  shall  presently  des¬ 
cribe  a  class  of  events  which  we  will  call  "regular  events." 

(We  would  welcome  any  suggestlcxis  as  to  a  more  descriptive  term.  ) 

We  assume  for  the  purpose  that  the  events  refer  to  the 
inputs  up  through  time  £  on  a  set  of  k  input  neurons 
the  same  for  all  events  considered;  but  the  definition 
applies  equally  well  for  any  k  >  1  or  even  for  k  0. 

The  events  can  refer  to  the  value  of  £.  Our  objective 
is  to  show  that  all  and  only  regular  events  can  be  represented 
by  nerve  nets  or  finite  automata.  We  have  already  seen  in 
Sect.  6.3  why  reference  to  the  time  is  called  fpr;  but  it  may 
be  illuminating  to  consider  some  examples  from  the  polht  of 
view  of  solving  given  nets. 

Consider  first  the  net  of  Fig.  22  taken  by  Itself  (the 
inhibitory  endbulb  from  Li  is  superfluous  now).  The  condition 
for  Li's  firing  (under  the  assumption  that  it  is  fired  initially), 
l.e.,  the  event  represented  by  Li's  firing,  is  given  by  formula 

J.i.(£)  ^-1  £  f  3. 

^McCulloch  and  Pitts  use  a  term  "prehensible, "  Introduced 
rathw  ^'ffar^ntly;  but  ?inc?e  we  did  not  understand  their 
deflhii  -r. .  w?  ashi  hesil  ant  ic  vh-  tei’in. 
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In  defining  "regular  events  "  we  shall  build  on  the  notion 
of  a  definite  event  of  length  as  originally  Introduced  In 
Sect.  5.1  and  completed  by  adding  £  >  ^  to  the  definition  in 
Sect.  6.3* 

But  now  we  extend  the  class  of  definite  events  by  providing 
that  the  description  of  such  an  event  may  also  contain  the 
specification  that  the  first  moment  is  1;  l.e.,  that 
i.e.,  that  £  =  X-  Events  with  this  specification  we  call  Initial 

SO  now  there  are  2®  definite  events  of  length  ^  with  k 

input  neurons,  namely  the  2®  as  before  (non— initial)  and 

the  2®  initial  ones. 

Now  each  event  we  shall  build  up,  starting  from  (and 
Including)  the  2®  definite  events  for  each  £,  will  be  inter- 

pretable  in  the  following  way.  The  statement  that  the  event  has 
occurred  (ending  at  time  £)  is  equivalent  to  the  statement  that 
one  of  a  non-empty  finite  or  infinite  class  of  definite  events 
has  occurred  (ending  at  time  £) .  (More  precisely,  the  class  may 
be  infinite  if  the  value  of  £  Is  unknown,  but  for  a  fixed  value 
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of  p  there  are,  of  course,  only  finitely  many  events  which  could 
have  occurred.) 

Our  class  of  regular  events  will  be  defined  inductively, 
starting  with  the  definite  events,  and  using  three  operations 
EVP,  EF,  E*F;  l.e.,  it  shall  be  the  least  class  containing 
the  definite  events,  and  closed  under  these  operations. 

Given  any  events  E  and  F  already  built  up  from  definite 
events  by  zero  or  more  applications  of  the  operations,  by  the 
event  EVF  we  shall  mean  the  event  which  occurs  if  E  occurs  or 
F  occurs.  In  other  words,  the  class  of  definite  events  which 
can  constitute  an  occurrence  of  E  V'F  is  the  sum  of  the  respec¬ 
tive  classes  for  E  and  F. 

Clearly  the  operation  is  associative;  l.e.,  (EVF)VGS 
EV(F\/G);  so  the  parentheses  can  be  omitted.  The  reason  for 
writing  equivalence  here  with  four  bars  will  appear  in  Sect.  7.;5. 

For  example,  if  E  and  F  are  definite  events  of  durations  ^ 
and  m,  respectively,  then  EVF  is  an  event  which  occurs  exactly 
when  an  event  (of  length  X)  belonging  to  E  occurs,  or  one  (of 
length  m)  belonging  to  F  occurs,  or  both.  One  might  be  tempted 
to  regard  this  as  a  definite  event  of  duration  max(2^,g);  but 
this  would  be  wrong,  since  supposing  ^  >  m  it  could  occur  when 
£  <  ^  =  max(2[,in),  namely  by  F  occurring.  Also,  this  would  not 
give  what  we  want  when  either  of  the  next  two  operations  is 
applied. 

Given  any  events  E  and  F  already  built  up  by  the  operations, 
by  EP  we  shall  mean  the  event  which  consists  in  E  having  just 
occurred  preceded  immediately  by  F  having  occurred.  Thus,  EF 
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.aas  'jo^urred  endlngitid  v  'Arr.er.  tne 

class  of  events  which  can  constitute  an  occurrence  of  E  (say  this 
one  is  of  length  has  occurred  in  time  £—^+1  to  £,  and  one 
of  those  which  can  constitute  an  occurrence  of  P  (say  this  one 
is  of  length  m)  has  occurred  in  the  period  £-^“ro+l  to  £— ^. 

(Note;  We  have  chosen  our  notation  EF  so  that  we  proceed  back 
into  the  past  in  reading  from  left  to  right.)  This  operation 
is  also  associative. 

For  example,  if  E,  F,  and  G  are  definite  events  of  lengths 
m,  and  n,  respectively,  einy  occurrence  of  (EVP)G  will  be  a 
definite  event  of  one  of  the  lengths  or  m+n.  (Say  2!  >  S* 

By  refraining  above  from  interpreting  E\/F  as  of  duration 
now  when  (EVP)G  occurs  by  P  occurring  ending  at  time  £,  the 
preceding  occurrence  of  G  must  end  at  time  £-m,  not  £-^. ) 

If  E  Is  an  initial  definite  event,  and  F  is  any  event  of 
our  class,  then  EF  is  an  event  which  never  occurs;  since  for  E 
to  occur,  its  first  moment  must  be  t  =  1;  so  \inder  the  interpre¬ 
tation  in  Sect.  6.3,  EFj^  Is  impossible  for  any  one  of  the 
definite  events  Pi,  Fg,...  vriiose  occurrence  can  constitute  an 
occurrence  of  F.  Thus,  in  this  case  EF  is  represented  by  the 
firing  of  P  in  the  net  of  Fig.  9. 

If  E  and  F  are  events  already  constructed,  then  by  E*F 
we  shall  mean  the  event  which  consists  df  zero  or  more  consecu¬ 
tive  occurrences  of  E  preceded  by  one  of  F.  That  is,  E*P  can 
occur  whenever 

n  times 


occurs  for  some  n  ^  0. 
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The  reason  we  do  not  define  E*  separately  as  a  unary  operation 
(expressing  that  E  has  occurred  consecutively  zero  or  more  times) 
Ipsteadcof  E*P  as  a  binary  operation,  is  that  then  for  n  =  0 
an  occurrence  of  E*  would  be  of  duration  0;  but  (at  least  for 
convenience)  we  are  requiring  the  lengths  of  our  definite  events 
to  be  always  >  1. 

To  say  that  E  has  happened  one  or  more  times  we  can  write 

E*E. 

For  example,  if  E,  F,  and  G  are  definite  events  of  lengths 
m,  and  n,  respectively,  an  occurrence  of  (E\/F)*G  must  be 
of  a  definite  event  of  one  of  the  lengths  +  to  +  n 

WS  A13.V  6  UWO  IbySv^iliS  Oi  riOt>d.'L>xOn  iOX" 

events:  (A)  logical  notations  for  definite  events  as  used  in 

Sect.  5  (with  the  addition  of  £  >  ^  in  Sect.  6.5  and  £  =  2! 
above)  and  for  some  other  events  in  Sects.  6.1  ff.;  (b)  our 
newly  introduced  notations  for  regular  events  starting  with 
single  capital  letters  as  representing  definite  events. 

There  will  be  ambiguity  if  we  use  (A)  as  the  starting 
point  for  (B)  instead  of  capital  letters,  unless  we  are  careful 
to  show  the  durations  of*  each  of  the  definite  events 

Ei,...,E  uBflid  as  the  units  for  the  construction  of  the  regular 

s 

events • 

The  question  of  translatablllty  between  the  systems  of 
notation  (A)  and  (B)  in  either  direction  has  not  yet  been 
examined  thoroughly. 
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HoweTei*,  we  have  verified  that  the  notations  (S)  can  he 
translated  into  notations  (A)  with,  of  course,  a  sufficient 
amount  of  mathematical  apparatus  added  to  the  logical  notations, 
fhe  details  are  technical  and  are  given  in  Appendix  t. 

But  it  ma^  he  instructive  now  to  give  a  few  simple  examples 
of  translation  in  the  other  direction,  i.e.,  from  (A)  to  (B), 

The  conventions  regarding  parentheses  are  those  of  algehra 
with  iVf,  BP,  and  I*F  analogous  to  ^f,  ef ,  and  e®f . 

Also,  the  associative  law  (S*PXd  ^  i^(Fd)  permits  omitting 
parentheses,  as  well  as  the  two  associative  laws  already  mentloniid 

The  event  of  duration  ^  which  happen#  for  all  Inputs  over 
the  Interval  to  £  we  call  the  identical  event  of  lentf^  ^|- 

for  ^  =  1  we  write  it  as  I,  then  in  general  ^  I  . . .  I  to  ^ 
factors ) . 

Let  the  result  of  adding  «  1  to  the  specif ioations  for 
a  definite  event  B,  to  mate#  It  initial,  he  wrlttem  1^* 

For  any  event  1  of  length  1  the  negation  S  Is  also  definite 
of  length  1. 

For  the  present  Illustrations,  let  the  event  of  length  1 
that  M  fires  at  time  g  (in  symbol#,  K{£)  be  written  simply  N; 
that  S  fires  simply  Kj  and  that  both  K  and  B  fire  Imi  written  L. 

Wow  the  events  described  as  follows  in  the  l#fii  coluam  are 
expx^ssed  by  the  corresponding  notations  In  the  rlj^t  column. 


(See  next  page) 
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(5i)t  i 

I*N 

( %  ^  /(t) 

N*N° 

(B)u  <El<H)^(i)u  <  t 

N*K 

*— ’h  Hi  ^<^3 

n*l°Vn*lk*^ 

p 

N(^)  for  at  least  two  values  of  jfc  <  £ 

I*NI*N 

N(t)  for  exactly  one  value  of  <  £ 

IT*rPvi?*NF*!7®, 

call  this  M 

N(t)  for  an  odd  number  of  values  of  ^  <  £ 

(F*nIT*N)*M 

£  >  3 

I3 

£  =  1 

1° 

£  =  1  mod  5 

£  <  3 

7.2  An  algebraic  transfomatlon;  We  list  several  equiva-^ 


lences : 

(1) 

(E  VF)  VG 

ev{fVg).  j 

(2) 

(EP)G 

E(FG).  ( 

Associative  laws 

(3) 

(E*P)G 

E*(PG)- 

(4) 

(E  V'F)G 

EGVFG. 

(5) 

E(P  VG) 

EFVEG.  S 

Distributive  laws 

(6) 

E*(F  VG) 

E*F  V  E*G.  J 

(7) 

E*F 

F  VE*(EF) . 

(a) 

E*F 

FVE(E*F). 
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Under  the  definition  just  given,  each  regular  event  is 
obtained  by  building  it  up  from  certain  definite  events  as 
the  units  by  zero  or  more  applications  of  the  three  operations. 
Of  course,  these  constructions  are  by  no  means  tinlque. 

Lemma  1.  For  any  £  >  2:  Every  regular  event  can  be 
expressed  as  a  finite  disiimction  of  one  or  more  regular  events, 
each  of  which  is  either  definite  of  length  <  s  or  is  an  event 
constructed  out  of  units  each  of  length  £  £•  (Also  true  tri¬ 
vially  for  £  =  1.) 

Of  course,  we  can  always  understand  there  is  at  most  one 
of  the  latter,  since  any  disjunction  of  them  is  again  one. 

We  write  out  the  proof  for  the  case  £  =  2. 

^e  lemma  is  true  when  the  given  event  is  definite;  then 
there  is  just  one  tern  in  the  disjunction,  which  is  of  the 
first  or  the  second  kind  according  as  its  length  is  1  or  more. 

Likewise,  if  E  and  F  each  have  the  property  described  in 
the  lemma,  so  does  EvF. 

Now  say  E  and  F  are  as  described,  and  consider  EP.  By 
use  of  the  distributive  laws  (4)  and  (5)»  EF  is  equivalent  to 
a  disjunction  of  terns,  each  of  which  has  one  of  the  following 
forms 

gipi,  g(a)p(^)^ 

where  In  each  case  ^  indieateit  a  definite  terra  of  length  1,  and 
indicates  a  tem  composed  out  of  units  each  of  length  >  2. 

Now  a  term  E^F^  can  be  construed  as  a  definite  event  of 
length  2;  so  it  is  of  the  second  kind  for  the  theorem  upon  con¬ 
sidering  it  as  one  of  the  units. 


RM-704 
Page  54 

Also  Is  of  the  second  kind. 

Now  consider  Using  (2),  ,  and  (4),  the  F^  can 

be  moved  progressively  Inward  until  finally  F^  occurs  only  In 
parts  of  the  form  HF^  where  H  Is  definite  and  of  length  >  2. 

Then  each  such  part  can  be  taken  as  a  unit,  which  will  be  of 
length  ^  3. 

For  the  last  form  we  proceed  similarly  using  (2) 

(from  right  to  left),  (5)  and  (8)  (in  combination  with  (5)  and 

(2)). 

Now  say  E  and  F  are  as  described  In  the  theorem,  and 
consider  E*P.  By  use  of  (6)  we  can  then  get  E*F  equivalent  to 
a  disjunction  of  terms  of  the  two  forms  E*P^  and  E*P^®^.  For 
Illustration  (noting  the  remark  just  following  the  theorem) , 
say,  e.g.,  E  Is  Ef  \/E|  \/E^®  ^ .  So  we  have  now  two  possibilities, 

(EiVEi  (Ei  Ve|v/e^2^)»p^2^. 

Consider  the  former.  This  Is  an  event  of  which  an  occurrence 
must  consist  of  one  occurrence  of  followed  by  n  >  0  occurrences 
of  various  of  the  events  Ei,  Eg  and  E^®^.  Let  Gi,...,Gg  be  all 
products  of  two  of  Ei,  B|,  and  E^®^j  l.e.,  Gi  Is  Elli,  Gg  Is 
EiEl,  etc.  Then  an  occurrence  of  the  event  Is  the  sa^  as  an 
occurrence  of  one  of  F^,  EiF*,  E|P^or  followed  by  zero 

or  more  occurrences  of  any  of  G^,...,Qq.  Thus,  In.  symbols  (and 
^Ustag  (6)  next  and.  then 
(Ei  V  E|  V  l^®^)»P^ 

S  (Gi  V. .  .VQg)*iP^  V  EiP^  V  E|P^  V 

=  (Gi  \/..  .Vg^)*p^  V  (Gi  V-  •  •  \/ag)*(iiP^  E|P^  V 
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SF'  VGg)*(Gx  V...  VGg)FW(Gi  V*.*  \/Gg)* 

(EiFVSiF^ 

Now  each  of  can  be  handled  as  was  one  of  E^P^,  E^®^F^®^ 

g(a)]|i^  gipCs)  ease  for  EF  above;  then  GiV...  V^G^  is 

composed  out  of  units  of  length  ^  2.  Then  by  the  method  for 
g(a)pi  case  for  EF  above,  (Gj  V . . .  VG^)?^  and  E^®^F^  are 

likewise,  while  EfF^  and  EjF^  are  definite  of  length  2.  So  the 
entire  expression  obtained  last  is  of  the  desired  fom.  Like 
arguments  apply  to  (Ei V E|  VE^® ^ .  Finally,  any  disjunc¬ 
tion  of  expressions  of  the  desired  formi  is  of  the  desired  form. 

7.3  Identity  and  equivalence;  In  dealing  with  regular 
events,  special  care  is  necessary  to  distinguish  between  senses 
of  "equivalence.”  As  we  introduced  them,  any  regular  event 
is  Identified  with  a  class  df  definite  events;  and  two  regular 
have  thus  far  been  treated  as  equivalent  only  if  these  classes 
of  definite  events  for  the  two  are  the  same. 

An  event  is  a  partition  of  all  the  possible  inputs  over 
the  whole  past  for  the  nerve  net  into  two  classes,  those  inputs 
for  which  the  event  occurs,  and  those  for  which  it  does  not 
occur. 

What  we  have  called  a  "non— Initial  definite  event  of 
length  is  a  partition  of  all  the  pasts  for  the  net  into  two 
classes,  such  that  all  pasts  of  length  <  ^,  i.e.,  for  which 
£  <  ^,  are  in  the  second  class  (those  for  which  the  event  does 
not  occur),  while  those  pasts  of  length  >  ^  are  in  the  first 
or  the  second  class  according  as  the  input  over  the  last  ^ 


rM_704 
Page  5® 

moments  has  or  has  not  a  certain  property;  i.e.,  the  classi¬ 
fication  is  independent  of  the  input  prior  to  £-^+1* 

But  could  two  non— Initial  definite  events  which  are  dls— 
tlrtct  in  the  value  of  ^  or  the  property  over  p— ^+1  to  £  be  the 
same  as  events? 

They  could  in  one  case,  namely  when  the  property  over 
£—^+1  to  £  is  impossible  of  occurring;  this  is  the  case  which 
was  treated  by  Fig.  9  in  Sect.  5*2.  These  definite  events  of 
length  X  various  values  of  ^  >  1  are  all  the  same  as  events. 
We  may  call  this  event,  which  never  occurs,  the  improper  (or 
impossible)  event. 

Outside  of  this  exception,  an  event  can  be  a  definite 
event  in  only  one  way.  For  suppose  we  have  an  example  of  an 
input  over  time  1  to  £  for  which  the  event  occurs.  Then  we 
may  seek  the  least  X  ^  R  such  that  the  event  also  occurs  when 
the  value  of  p,  is  changed  to  X  3-*^^  input  over  time  1  to  ^ 

(=  p)  is  what  it  was  formerly  over  time  £—^+1  to  £.  This  X  si^st 
be  the  length  of  the  event;  and  the  property  of  the  input  over 
the  last  X  moments  v^ich  defined  the  event  is  obtained  by  eon— 
sidering  what  inputs  over  this  time  give  occurrences  of  the  event. 

Similar  remarks  apply  to  "initial  definite  events  of 
length  X’**  Here  all  pasts  for  which  -g  ^  X  second 

class.  The  initial  definite  events  of  length  X  which  never 
oc-«iir  are  Mil  ~ 

Coffi'isihiai  tb®  of  nea^isitiai  nad  inlbl&l  defl.- 

nite  events,  atn  event  can  be  a  definite  event  in  only 
one  nfn-4,nlfcinl.  or  initial,  hut  rtot 


RM-704 
Page  57 

both,  with  only  one  length  and  with  only  one  property  of 
the  Input  over  £-^+1  to  £),  except  for  the  Improper  event  which 
can  be  construed  as  either  non— Initial  or  initial  and  with 
any  So  actually  there  are  only  2®  ^  -  1  distinct  definite 

events  of  length  ^  with  k  input  neurons;  in  Sect.  7.1  we  counted 
the  improper  event  twice,  once  as  a  non— initial  and  once  as 
an  initial  event.  For  k  *  0,  there  are  thus  just  5  events  of 
length  ^  ,  the  possible  non-initial  one  I^,  the  possible  initial 
one  and  the  Impossible  one  T. 

Now  consider  a  regular  event  of  the  form  EVP,  where  E 
and  F  are  definite  of  length  Quite  evidently  the  E  and  F 
are  not  uniquely  determined  from  the  event.  For  example,  there 
might  be  two  k  x  ^  tables  exactly  for  which  1  occurs,  a  third 
for  which  F  occurs.  By  recombining,  taking  Ei  as  occurring  when 
the  first  table  applies,  and  Fi  when  the  second  or  the  third 
applies,  we  get  the  event  as  EiVFi  with  different  components, 
or  indeed,  the  event  can  be  considered  as  one  event  of  length 
Now,  in  fact,  our  transfomatlons  of  events  in  Sect.  7.2 
were  such  as  to  preserve  the  class  of  definite  events  under¬ 
lying  a  given  regular  event,  except  that  sometimes,  e.g., 

was  reeonstrued  as  a  definite  event  of  length  2.  To  make 
it  exact  what  transformations  shall  be  allowed,  we  can  reconsider 
a  regular  event  as  given  by  saying  which  of  various  tables  of 
length  ^  for  various  with  or  without  specification  that 
p  ^  (rather  than  merely  P  ^  would  describe  an  occurrence 
of  it.  For  if  it  is  given  by  saying  which  of  various  definite 
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events  would  describe  an  occurrence  of  it,  then  we  can  replace 
each  of  these  by  the  tables  (zero  or  more  up  to  2^  of  them) 
which  constitute  an  occurrence  of  the  respective  definite  event. 
We  shall  say  that  two  regular  events,  as  given  in  the  notations 
of  Sect.  7.1  starting  with  specified  definite  events,  are 
identical  if  the  resulting  classes  of  k  x  ^tables  (for  various 
are  the  same.  W®  write  iden'clty  by  . 

The  empty  class  of  k  x  ^  tables  goes  with  the  improper 
event;  call  this  event  T.  We  have: 

(9)  EVT  ^  T  VE  ^  E. 

(10)  ET  ^  TE  s  T. 

(11)  E*T^  T. 

(12)  T*1  m  E. 

These  permit  simplifications  of  events  into  which  T  is  built;  in 
fact,  all  T's  can  be  removed,  unless  the  whole  becomes  T. 

Now,  unfortunately,  given  an  event  as  simplsc  a  partition 
of  the  possible  inputs  over  the  virtiole  past  for  the  net,  the 
class  of  k  X  tables  in  terms  of  which  it  can  be  constructed 
as  a  regular  event  is  not  unique. 

Consider  the  example  of  NVNI*I  and  N,  where  k  =  1,  and 
N  signifies  the  event  of  length  1  consisting  6f  the  firing  of 
the  one  input  neuron  N  at  time  p. 

The  only  k  x  ^  table  for  N  is  that  having  a  1  in  its  one 
position.  But  NVNI*I  has  this  table,  also  both  tables  of 
length  2  agreeing  in  the  first  row,  also  all  four  tables  of 
length  3  agreeing  in  the  first  row,  etc. 
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lii#  N  V  NI*I  and  N  each  occur.  If  and  only  If  the  input 
neuron  fires  at  time  £;  so  as  events  they  are  the  same.  We 
call  this  sameness  equivalence ,  and  write  N\/N*I  H  N. 

The  importance  of  the  distinction  is  that  from  E  =  F  we 
can  infer  EG  ^  FG,  GE  ^  GF,  E*G  S  P*G,  and  G*E  S  G*B;  but  we 
cannot  make  the  first  and  thiM  inferences  in  terms  of  equiva¬ 
lence  =  .  In  particular,  N  VNI*I  =  N,  but  not  (N  n/nI*I)NS  NN. 

(Of  course,  ES  F  Implies  1  S  F;  but  not  conversely.)  As  another 
example,  but  I*IN  ^  I»I°N  =  T  (by  (I5)  below  and 

(11)). 

Summarizing,  our  theory  of  regular  ”events,"  with  our 
operations  EVF,  EF  and  E*P  and  the  relations  apply  to  classes 
of  k  X  ^  tables  (fixed  k  and  varying  £)  in  terms  of  which  we  can 
represent  the  events,  rather  than  to  the  events  in  the  simple 
sense.  More  particularly,  it  is  the  two  operations  EF  and  E»F 
for  which  the  class  of  tables  for  E,  rather  than  merely  the 
resulting  event  E,  must  be  known,  because  the  lengths  of  the 
tables  enter  into  the  meaning  of  the  operations. 

It  would  thus  be  more  explicit  to  say  that  we  are  dealing 
with  a  theory  of  certain  expressions  for  events  ("representa¬ 
tions"  would  be  a  good  word,  if  we  were  not  using  it  already  in 
another  sense). 

We  now  extend  our  notion  of  "prepositive"  to  initial  events, 
by  saying  that  all  except  the  improper  one  (which  is  at  the  same 
time  non— initial  and  as  such  prepositive  under  the  definition 
in  Sect.  6.5)  are  not  prepositive .  Single  k  x  ^  tables  are 
special  cases  of  definite  events  of  length  so  the  definition 
applies  to  them. 
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Now  we  say  that  a  regular  event  as  given  by  a  class  of 
k  X  ^  tables  (fixed  k  and  varying  is  prepositive,  if  all 
the  tables  of  the  class  are  prepositive. 

In  Sect.  6.5  we  saw"that  prepositiveness  was  necessary 
an(f  suffislerit  a  giarva  a#;;  with  all  Inns]’  neurons  ini¬ 

tially  quiet  constructed  to  represent  a  non— initial  definite 
event  of  length  ^  when  £  >  ^  should  also  represent  it  correctly 
(without  "hallucinations")  when  £  <  ^. 

The  extension  to  initial  definite  events  preserves  this 
as  a  necessary  and  sufficient  condition  for  representablllty 
wltb  all  inner  neurons  initially  quiet;  the  necessity  is  clear 
by  reasoning  similar  to  that  in  Sect.  6.5,  and  the  sufflcifncy 
holds  because  there  is  no  such  prepositive  event  except  the 
Impossible  one. 

Furthermore,  now  a  sufficient  condition  that  in  represen¬ 
ting  an  event  it  be  possible  to  take  all  inner  neurons  quiet 
initially  is  that  there  be  a  way  of  expressing  the  event  in 
terms  of  definite  events  and  our  three  operations  for  which  the 
corresponding  class  of  tables  (or  of  definite  events)  is  pre¬ 
positive.  This  will  be  Included  as  part  of  the  next  theorem. 

To  get  a  necessary  condition,  we  introduce  the  idea  of 
a  minimal  set  of  k  x  X  tables  (fixed  k  and  varying  ^)  for  an 
event.  Start  with  any  set  of  k  x  tables  for  the  event,  and 
to  each  table  consider  the  least  se^ent  of  it  ending  at  time  £ 
for  which  all  backward  extensions  describe  occurrences  of  the 
event.  Replace  the  table  by  this.  Carrying  out  the  process 
for  each  table  in  the  given  set,  we  get  a  minimal  set. 
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The  minimal  set  so  obtained  Is  unique  for  a  given  event, 
as  one  gets  the  same  minimal  set  by  first  extending  each  given 
table  to  an  Initial  table  in  all  possible  ways  (which  method 
gives  the  complete  set  of  tables,  which  Is  unique  for  the  event;); 
and  then  minimizing  this  (by  the  above  process  which  leads  to 
a  unique  result),  we  get  the  same  class  of  tables  as  in  minimizing 
dl rectly. 

Of  course,  the  method  of  minimizing Js  not  described 
"constructively,"  and  one  question  which  arises  at  once  is 
whether  a  constructive  minimization  process  for  a  set  of  tables 
corresponding  to  a  regular  event  as  expressed  in  terms  of  defi¬ 
nite  events  and  our  three  operations  exists. 

Another  question  is  whether  the  minimal  set  of  tables  must, 
for  a  regular  event,  necessarily  be  one  which  corresponds  to 
an  expression  for  the  event  in  terms  of  definite  events  and 
the  operations.  (The  complete  table  does,  as  will  follow  from 
the  proof  of  Theorem  6  in  Sect.  9*) 

We  do  not  go  into  these  questions,  which  one  would  naturally 
Investigate  if  the  study  is  to  be  continued. 

However,  we  can  now  say  that  a  necessary  condition  that  a 
regular  event  be  jrepresent&blt  by  a  net  with  all  inner  neui?ons 
initially  quiet  is  that  the  minimal  set  of  tables  for  it  be  pre¬ 
positive. 

Some  algebraic  simplifications  are  possible  when  initial 
definite  events  enter  into  an  expression  for  a  regular  event. 

Say  E®  is  an  initial  definite  event.  Then  for  any  regular  event  F 

(13)  E®P  ^  T. 

(14)  EO*f  S  T. 
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Used  along  with  (2)  —  (4)  (and  for  simplification  (9)  —  (12)), 
we  come  out  with  the  result  that  no  Initial  event  need  enter 
into  an  expression  for  a  regular  event,  other  than  as  an 
m'srit  in  ths  fnl'i-jwing  aensa. 

foi-  a,  for  a  :0e?gul3,r-  avsni  :ixi 

of  definite  events,  we  define  recursively  as  follows 
which  oceurrences  of  definite  events  in  it  are 

Iri  a  regular  event  given  as  simply  a  definite  event,  that 
definite  event  is  earliest. 

The  earliest  events  in  E  and  the  earliest  events  in  F  are 
the  earliest  in  EVF. 

The  earliest  events  in  F  are  the  earliest  in  EF  and  in  E*P. 

7.4  Representability  of  regular  events; 

Theorem  4;  To  each  regular  event,  a  nerve  net  can  be  con¬ 
structed  which,  when  started  in  a  prescribed  way,  represents 
the  event  by  firing  a  certain  inner  neuron  at  time  £  +2,  if  and 
only  if  that  event  has  occurred  ending  at  time  £  inclusive.  If 
the  given  event  is  prepositive,  the  representation  can  be  by 
a  net  started  with  all  inner  neurons  quiet. 

Proof  is  based  on  Lemma  1,  Sect.  7.2,  for  £  =  2.  The 
Theorem  is  true  for  T,  by  Pig.  9,  Sect.  5-2;  and  for  other  events 
by  Sect.  7.3,  we  can  exclude  T  as  a  unit. 

So  first  we  give  the  proof  for  the  ease  of  an  event 
(not  T)  constructed  out  of  units  (not  T)  each  of  length  2  or  more 
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This  we  do  by  induction  on  the  nvimber  n  of  occurrences 
of  units  in  the  expression  for  the  event. 

In  the  induction  we  will  arrange  at  each  stage  that  the 
neuron  which  is  to  fire  at  time  £+2  will  be  (as  in  Sect.  5.2, 
since  T  is  excluded)  one  of  threshold  1  impinged  upon  by  only 
excitatory  endbulbs  (l.e.,  it  effectuates  a  disjunction  opera¬ 
tion)  with  no  axons  feeding  )|iick  into  the  net. 

If  n  =  1,  then  the  event  is  a  definite  one  E.  We  have 
three  cases.  (a)  E  is  prepositive,  hence  not  initial.  The  net 
is  as  given  in  Sect.  5.2,  the  reasoning  that  this  net  works 
being  supplemented  as  in  Sect.  6.3.  (b)  E  is  not  initial  and 

not  prepositive.  We  use  the  treatment  given  in  Sect.  6.2 
employing  Fig.  21.  (c)  E  is  initial.  Then  we  use  an  inner 

neuron  as  follows,  treated  for  the  net  coastxrutfcien  of  Sect.  5.2 
as  though  it  were  an  additional  input  neuron  required  for  the 
occurrence  of  the  event  to  fire  at  time  g—^+1. 


Figure  24 

This,  of  course,  is  simply  a  neuron  whose  condition  for  firing 
is  £  =  1. 

Now  if  n  >  1,  the  event  under  consideration  is  of  one  of 
the  forms  eVF,  EF,  and  E»F  where  E  and  F  are  each  constructed 
from  <  n  units. 
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First,  suppose  the  event  is  EV/F.  Then  by  the  hypothesis 
of  the  induction  we  can  construct  nets  to  represent  E  and  to 
represent  F,  say  with  representing  neurons  P  and  Q,  respec¬ 
tively,  each  with  threshold  1  and  only  excitatory  endbulbs 
impinging,  and  with  no  axons  feeding  back.  To  represent  Ex/F 
we  "identify”  P  and  Qj  i.e.,  we  replace  them  by  a  single  neuron 
— call  it  P — having  all  the  endbulbs  which  separately  Impinged 
on  P  and  on  Q,  and  we  similarly  identify  the  input  neurons 
Ni,...,Nj^  for  the  two  nets,  i.e.,  the  axons  which  led  from 
Ni,...,Nj^  in  the  net  for  E,  and  those  in  the  net  for  F  now  both 
lead  from  Ni,...,Nj^.  The  construction  can  be  diagramed  as 
follows,  using  heavy  lines  to  represent  a  number  of  axons.. 


The  heavy  bundle  of  neurons  leading  to  P  from  the  left  are 
those  which  would  be  required  in  the  net  for  E  separately;  like¬ 
wise  from  the  right  in  the  net  for  F.  The  bundle  from  Ni  toward 
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the  left,  those  from  Ni  In  the  net  for  E,  to  the  right  In  the 
net  for  F,  etc.  The  fact  that  the  output  neurons  for  the  two 
given  nets  had  no  axons  leading  back  in  insures  that  they 
still  operate  independently  of  each  other  in  this  combination. 

Next  consider  an  event  EF.  In  the  construction  out  of 


units  which  we  are  using  for  E,  consider  those  occurrences  of 
units  in  it  which  are  earliest.  The  events  we  are  considering 
refer  to  k  input  neurons  Ni,...,Nj^.  Now  consider  the  event 

I  — 

E  which  is  obtained  from  E  by  modifying  each  earliest  unit  to 


make  it  refer  to  dne  new  neuron  which  is  required  to  fire 

at  the  second  moment  of  each  such  earliest  unit.  There  is 


such  a  second  naament  in  the  period  of  the  unit,  by  our  assump¬ 
tion  in  connection  with  the  use  of  Lemma  1  that  each  unit  is 
of  length  >  2.  Also,  the  resulting  event  1«  rsmlar  with  the 
same  number  of  occurrences  of  units,  since  this  change  in  the 
earliest  units  only  gives  an  event  with  the  same  structure  in 
terms  of  its  respective  components  by  the  operations  E'^F,  EF, 
and  E*F.  So  by  the  hypothesis  of  the  induction  on  n,  we  can 

I 

represent  this  event  E  by  a  net.  However,  we  simplify  the 
construction  by  leaving  out  the  neuron  of  Fig.  21  in  the  ease 

I 

of  earliest  events  in  E  which  come  under  Case  (b)  for  definite 
events.  (By  remarks  in  Sect.  7.4,  Case  (c)  can  be  excluded.) 

Now  the  net  for  EP  is  obtained  by  identifying  in  the 

I  — 

net  for  E  with  the  output  neuron  Q  of  the  net  for  F,  and  of 


course,  identifying  Ni 


as  input  neurons  for  the  two  nets. 


I 

The  output  neuron  is  that  for  E  .  The  construction  can  be 


diagramed  thus: 
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I 

The  event  E  Is  positive,  requiring  a  firing  of 

second  moment.  But  can  be  fired  only  at  a  time  later 

than  2,  since  in  its  role  of  output  neuron  for  the  net  for  F 


it  fires  at  time  ^2  (£  >  l)  where  ^  is  the  last  moment  of  an 

loccurrence  of  F.  No  ”haillucinafcioft*'  1$  possible  a#  a  result  c£ 


leaving  out  the  neurons  of  Fig.  21  for  the  units  in  E  which 
were  not  prepositive,  as  this  necessity  that  fire  at  the 

second  moment,  vdiich  must  be  >  2,  prevents.  (In  fact,  the 
arguments  of  Sect.  6.3  that  ’’hallucinations"  can  occur  when  an 


event  is  not  prepositive  do  not  apply  now,  since  some  inner 
neurons  of  the  net  for  F  will  necessarily  be  firing  at  the 
first  moment  of  these  units  of  E  . )  These  remarks  (with  the 
avoidance  of  the  neuron  of  Fig.  21)  are  necessary  to  establish 
the  last  remark  of  the  theorem. 


We  have  lastly  the  case  for  1*F.  As  in  the  preceding 

I 

case  we  modify  E  to  E  .  Then  we  combine  the  nets  oDbtained  by 
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the  hypothesis  of  the  induction  (omitting  Pig.  21  in  treating 

t . 

earliest  units  in  1  J  as  diagramed  thus: 


Under  the  assu^tion  of  infinite  past  ti^  (as  in  Sect.  6.1), 
the  firing  of  P  could,  of  course,  be  explained  by  E  having 
occurred  repeatedly  ad  inflnittm  into  the  past.  But  here  we 
are  understanding  that  the  whole  net  is  started  in  a  certain 
condition,  which  is  either  that  all  inner  neurons  are  quiet  or 
that  acme  inner  neuron  (or  neurons,  if  we  prefer)  in  the  net 
for  P  are  fired,  according  as  F  (and  therefore  1*F)  is  preposi¬ 
tive  or  not.  Then  as  in  the  reasoning  under  the  treatment  of 

1 

IP,  the  net  for  E  can  only  be  a  cause  of  P  and  firing 

if  there  has  originally  been  a  firing  derived  from  the  net  for 

I 

F,  which  serves  as  an  input  into  that  for  E  that  must  be  fired 
at  the  second  moment  for  the  latter.  Of  course,  thereafter  P 
will  be  fired  on  each  repetition  of  E.  (P  and  must  be 

separate,  to  meet  the  condition  that  the  output  neuron  not 
feed  back  into  the  net.) 
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This  completes  the  treatment  for  regular  events  construc¬ 
ted  from  units  each  of  length  >  2. 

Now  by  Lemma  1,  and  using  the  method  already  Indicated 
for  the  basis  (l.e.,  f^r  n  »  l)  to  treat  definite  events  of 
length  1,  and  the  method  already  used  under  the  case  of  the 
induction  step  (n  >  l)  for  EV^F  to  combine  nets  for  different 
disjunctive  members,  we  get  the  theorem  for  regular  events  In 
general. 

7.5  Discussion  of  the  proof  and  further  problems:  As 
we  have  already  remarked  In  Sect.  6.2,  the  use  of  a  net  with 
Initially  fired  Inner  neurons  seems  xmnatural.  But  this  Is 
unavoidable.  If  we  are  to  represent  non-preposltlve  events, 
since  we  must  (by  examples  such  as  are  given  at  the  beginning 
of  Sect.  7.1)  make  our  mathematical  theory  complete.  A  way 
of  avoiding  the  use  of  such  nets  biologically,  namely  by  con¬ 
sidering  only  events  dated  from  some  environmental  stimuli, 
has  been  Indicated  (Sect.  6.2). 

A  second  respect  In  which  the  present  proof  seems  arti¬ 
ficial  and  leads  to  complicated  nets  Is  In  the  use  of  Lemma  1, 
the  proof  of  which  Involves  rather  extensive  reformulation  of 
the  events. 

If  we  deal  only  with  events  which  are  already  expressed 
In  terms  of  units  each  of  leng^  >  t ,  jor  have  the  form  for 
Lemma  1,  the  proof  of  the  theorem  Is  straightforward  and  the 
nets  constructed  are  simple,  l.e.,  of  a  degree  of  complexity 
corresponding  very  well  to  the  complexity  of  the  given  descrip¬ 
tion  of  the  event  to  be  represented. 
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The  necessity  of  using  Lemma  1,  if  we  do  not  restrict 
the  events  to  be  already  expressed  in  the  foimi  for  that  lemma, 
arises  from  the  fact  that  in  Sect.  5  we  needed  a  lag  of  2  to 
represent  definite  events  in  general. 

Some  simple  events  can  be  represented  with  lag  of  Ij  and 
for  these  the  units  into  which  we  feed  the  outputs  from  the 
nets  representing  the  preceding  events  would  not  need  to  be 
of  length  >  2.  Thus,  to  give  a  uniform  treatment  in  proving 
the  theorem,  we  resorted  to  a  device  (the  proof  of  Lemma  l) 
or  a  restriction  (that  the  property  of  Lemma  1  is  already  present) 
which  can  be  dispensed  with  in  many  particular  cases.  This  is 
why,  e.g.,  in  Sect.  6.2.,  simpler  nets  are  available  for  repre¬ 
senting  certain  indefinite  events  than  would  be  given  by  the 
method  of  proof  of  the  present  theorem. 

As  was  noted  in  Sect.  often  definite  events  can  be 

represented  by  simpler  nets  by  using  a  lag  greater  than  2.  Then 
the  method  of  net  construction  for  the  proof  of  Theorem  4  would 
require  the  use  of  Lemma  1  for  ^  =  the  greatest  lag  used  in 
any  of  the  units  (or  possibly  not  this  great,  depending  on  how 
the  imits  enter).  As  the  proof  of  Theorem  4  is  given,  also  the 
net  would  have  to  be  chosen  so  that  the  representing  neuron 
appears  as  in  Fig.  2;  i.e.,  performs  a  disjunction  operation. 

This  could  always  be  arranged  by  an  increase  of  1  in  the  lag. 

But  also  probably  the  proof  can  be  adapted  to  apply  directly, 
somewhat  as  the  proof  of  Theorem  1  was  generalized  to  get 
Theorem  2  (but  we  have  not  examined  this  in  detail). 
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The  difficulty  which  calls  for  Lemma  1  arises  when  we 
try  to  represent  a  repetition  of  some  event  which  is  shorter 
than  the  time  necessary  for  the  net  to  organize  a  representa¬ 
tion  of  it  by  the  firing  of  a  single  neuron  the  solution  by 
Lemma  1  consists  in  considering  grosser  events  before  attemp¬ 
ting  to  represent  repetitions  of  them. 

If  we  consider  that  one  or  two  synaptic  delays  are  pro¬ 
bably  not  significant  for  determining  behavior  in  an  organism 
(as  we  remarked  at  the  end  of  Sect.  6.3),  it  seems  that  the 
complication  is  connected  with  an  over— refinement  in  our  model 
of  the  biological  reality. 

So  we  can  urge  that  the  methods  of  net  construction  used 
in  proving  the  theorem  are  simple  enough,  granting  that  from 
the  general  method  we  can  often  start  out  to  find  simpler  nets 
in  special  eases. 

The  question  may  occur  to  the  reader,  why  did  we  select 
the  particular  three  operations  E  VP,  EP,  and  E*P?  When  we 
say  that  the  net  constructions  are  simple,  we  meein  simple  for 
events  already  described  from  definite  events  by  use  of  these 
operations. 

A  pressing  problem  now  is  to  consider  what  kinds  of  events, 
described  originally  in  other  terms,  can  be  described  in  these 
terms;  and  so  eventually  what  kind  of  behavior  can  be  explained 
on  the  basis  of  nerve  net  control. 

This  is  a  problem  one  would  natui^lly  investigate  in  detail 
next.  We  have  not  done  so  thus  far,  since  this  report  is 
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intended  only  to  reduce  to  writing  the  author’s  thinking  on 
the  subject  during  August  1951 »  and  not  to  try  to  carry  the 
investigation  further,  except  for  the  minimum  amount  of  fillir® 
in  details  which  was  unavoidable  in  the  process  of  writing. 

However,  it  is  very  plausible  that  the  notation  for 
regular  events  in  terms  of  definite  events  combined  by  the 
three  operations  will  p3?ove  handy  in  describing  eyents.  The 
simple  examples  given  at  the  end  of  Sect.  7.lT,  and  some  others 
slightly  more  complicated,  encourage  this  hope. 

On  the  other  hand,  given  a  description  of  an  event  in  teims 
of  definite  events  and  the  three  operations,  it  will  in  some 
cases  be  difficult  to  see  what  the  event  consists  of;  we  know 
of  eases  in  which  a  very  complicated  description  is  actually 
equivalent  to  a  much  simpler  one.  (This,  in  fact,  is  usually 
the  case  for  descriptions  provided  by  the  method  of  proof  of 
Theorem  6  in  Part  II.) 

So  there  are  problems  of  translatability  in  both  directions 
between  the  notations  events  e.ftd.  other  nttatlong  for 

events  or  descriptions  of  events  in  ordinary  language.  These 
problems  have  so  far  been  touched  only  superficially,  and  are 
crucial  for  determining  how  far  the  present  results  carry  us 
toward  practical  general  techniques  for  construction  of  nets 
for  given  purposes. 

These  questions  are  related  to  questions  about  transforma¬ 
tions  between  different  expressions  for  the  same  event  in  terns 
of  our  operations.  Can  we  obtain  any  normal  forms,  i.e., 
simplest  fo3?ms  or  convenient  standaixi  forms,  for  descriptions 
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of  regular  events,  to  which  given  forms  are  equivalent  In  the 
sense  of  Sect.  7.3?  Is  there  any  decision  procedure  for  the 
equlv^iehae  of  two  expressions  for  events  (in  the  technical 
sense  of  modern  logic)?  These  questions  are  closely  related 
to  questions  raised  In  Sect.  7.3* 

Similar  questions  apply  to  Identity  In  the  sense  of 
Sect.  7.3;  but  equivalence  Is  the  Important  relation  for  the 
applications  of  the  theory. 

These  questions  are  partly  algebraic  In  character.  Some 
questions  are  also  raised  in  Part  II  and  Appendix  2. 

Success  in  reducing,  to  terms  of  definite  events  and 
EVF,  EF,  and  1*F,  events  as  expressed  in  ordinary  language 
or  as  they  arise  In  explaining  organic  behavior  or  creating 
robots  for  prescribed  purposes  would,  of  course,  give  a  justi¬ 
fication  for  our  selection  of  the  operations. 

Our  actaltl  reason  for  selecting  them  Is  that  (as  was  men¬ 
tioned  In  Sect.  7.1)  a  converse  of  Theorem  4  will  be  proved 
In  Part  II. 

Thus,  every  event  which  can  be  represented  must  be  expres¬ 
sible  in  tffimis  of  E^F,  EF,  and  E*P,  starting  from  definite  events. 

In  particular,  we  have  thus  demonstrated  that  McCulloch- 
Pitts  neurons  can  govern  any  kind  of  behavl0r  which  any  other 
kind  of  digital  automaton  at  all  can  govern.  This,  of  course, 
includes  a  number  of  special  results  which  they  obtained  for 
alternative  kinds  of  nerve  nets,  but  is  more  general. 
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Having  been  first  led  to  the  three  operations  in  connec¬ 
tion  with  the  converse  of  Theorem  4  (i.e..  Theorem  6  in  Part  II), 
it  was  natural  to  see  whether  the  present  theorem  would  hold. 

But,  of  course,  the  fact  that  our  ttiree  operations  are 
completely  general  (by  Theorem  6)  does  not  settle  the  question 
whether  they  will  prove  to  be  a  convenient  and  practical  way 
to  deal  with  events.  Possibly  some  other  selection  will  prove 
to  be  more  convenient.  Or,  we  may  add  other  operations  and 
express  these  in  turn  in  terms  of  our  three. 

7.6  Conjunction  and  negation;  We  did  not  include  the 
operations  ^  (and)  and  (not)  in  our  definition  of  regular 
events,  because  in  the  converse  theorem  (Theorem  6)  we  do  not 
need  to. 

In  this  section  we  will  show  that  net  constructions  can 
be  managed  60  that  the  two  operations  can  be  included.  However, 
we  will  only  treat  them  when  applied  to  events  already  repre¬ 
sented  by  nets,  and  we  will  not  thereafter  use  EP  and  E*F. 

Prom  the  converse  theorem  it  will  follow  that  any  events 
we  thus  express  using  also  ^  and  must  be  expressible  without 
them.  But  the  definitions  obtained  in  this  way  are  very  com¬ 
plicated,  and  simple  definitions  do  not  appear  to  be  immediately 
forthcoming.  (But  we  have  not  examined  the  pixsblem  thoroughly. ) 

We  are  not  attempting  to  use  2^  and  inside  EF  and  E*P 
(except  in  the  original  constructions  of  definite  events  as 
the  imits)  since  we  have  not  set  up  a  representation  of  these 
operations  in  terms  of  classes  of  definite  events  or  of  k  x  ^ 
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tables.  It  does  not  seem  to  be  iinraedlate  what  is  the  best 
way  to  do  this. 


Theorem  Eaeh  event  construe tlble  from  regular  events 


Proof:  Say  the  event  Is  eonstrueted  by  the  operations  of 
the  propositional  calculus  from  certain  expressions  for  regular 
events.  Consider  any  one  of  the  latter.  Wherever  a  part  occurs 
in  it  of  the  form  E*F,  replace  this  by  F\/E(E*P)  using  (8). 

After  this,  apply  (4)  and  (5)  whenever  possible.  Using  also 
(2)  if  necessary,  we  are  thus  led  to  an  expression  for  the 
original  event  by  operations  of  the  propositional  calculus  in 
terms  of  regular  parts  of  the  forei  Ei(...E^)  where  Ei  is  defi— 
nitej  for  this  purpose  we  take  the  V's  which  have  been  brought 
outermost  in  the  expressions  for  the  regular  events  as  part  of 
the  construction  in  terms  of  the  operations  of  the  propositional 
caleiilw©.  Hay  there  m  Bijtch  partB^  bajj,  ihetr  first  factors 

El  i-"'  (  i  =»  1, , . . ,«)  >  and  the  iidiole  expressions  El—'  ( . .  •'&„—*  )  • 

IL(i) 

Let  be  or  Ei^i^  according  as  is  >  1  or  =  1, 

?diere  '  has  the  meaning  given  it  in  the  proof  of  Theorem  4.  Now 
we  can  take  exactly  the  same  combination  by  operations  of  the 
propositional  calculus  of  Ei^*^",  ...  ,  that  the  given 

event  Is  of  li ^ . .E^J ^ ^ ,  ...  ,  ^ ) .  This  can 
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be  ti*eated  as  a  definite  event  of  length  equal  to  the  greatest 
length  of  any  of  its  components,  and  a  net  can  be  constructed 
for  it  by  Theorem  1,  with  input  neurons  and  for 

each  1  for  which  2.^)  >  1  a  neuron  required  to  fire  at 

time  event  to  occur.  Feeding  the  outputs 

from  the  nets  for  ...  ^  appropriately  into  this.  Instead 

2.(1) 

of  as  before  into  respective  nets  for  e(— ^  ’ ,  we  get  a  net  for 
the  event  in  question. 


PART  II  —  FINITE  AUTOMATA; 

8.  The  Concept  of  a  Finite  Automaton; 

8.1  Cells:  Time  shall  consist  (as  in  Sect.  5  ff.)  of  a 
succession  of  discrete  moments  numbered  as  integers.  We  shall 
mainly  be  concerned  with  the  ease  of  only  positive  integers,  as 
in  Sect.  6.2  ff,  but  will  consider  the  case  of  all  the  integers 
in  Appendix  1. 

We  shall  consider  automata  cons tme ted  of  a  finite  number 
of  parts,  each  being  capable  of  a  finite  n\imbeif  ^  2  of  states 
at  any  given  moment.  Call  these  parts  cells. 

We  shall  distinguish  two  kinds  of  cells,  input  cells  and 
inner  cells.  Say  there  are  k  input  cells  and  m  inner  cells. 

An  input  cell  admits  2  states  0  and  1  (or  “quiet”  and 
’’firing”),  which  we  consider  to  be  deteiroined  by  the  envirorment . 

This  restriction  to  2  states  for  input  cells  is  to  make 
the  notion  of  an  input  to  the  automaton  coincide  with  the  notion 
of  input  to  a  nerve  net  as  formulated  in  Sect.  4.  But  the 
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present  theory  would  work  equally  well  with  more  than  2  states. 
Nothing  is  gained  thereby,  however,  as  £  cells  each  admitting  2 
states  could  be  used  to  replace  one  cell  admitting  any  number  £ 
(2  <  £  <  22.)  states  0,  1,  ...  ,  £-1,  where  if  £  <  22  we  could 
consider  only  inputs  in  which  states  £,  ...  ,  22-1  do  not  occur 
or  identify  all  of  these  states  with  the  state  £—1  in  all  the 
operations  of  the  automaton. 

The  nvimber  of  states  of  an  inner  cell  is  not  restricted 
to  2,  and  different  inner  cells  may  have  different  numbers  of 
states . 

Say  the  input  cells  are  Ni,...,^  (k  >  0);  and  the  inner 
cells  are  (m  >  l),  with  respective  numbers  of  states 

£i , . . .  (each  >  2) . 

The  state  of  each  inner  cell  at  any  time  Jb  is  determined 
by  the  states  of  all  the  cells  at  time  -1.  Of  course,  it 
may  happen  that  we  do  not  need  to  know  the  states  of  all  the 
cells  at  time  _^1  to  infer  the  state  of  a  given  inner  cell  at 
time  Jb.  Our  formulation  merely  leaves  it  unspecified  what 
kind  of  a  law  of  determination  we  use,  except  to  say  that 
nothing  else  than  the  states  of  the  cells  at  t— 1  can  matter. 

If  time  is  given  as  beginning  with  t  =  1,  the  state  of  the 
inner  cells  at  that  time  is  to  be  specified. 

A  particular  example  of  a  finite  aut^naton  is  a  McCulloeh— 
Pitts  nerve  net.  Here  all  the  cells  have  just  2  states,  and 
the  principles  stated  in  Sect.  :5,  together  with  the  arrangement 
of  axons  and  the  kind  of  endbulbs  on  each  case,  give  the  law 


determining  the  state  of  each  Inner  neuron  at  time  ^  from  the 
states  of  all  the  neurons  (or.  In  fact,  from  only  those  having 
endbulbs  synapslng  on  the  given  one)  at  time  ^1. 

Another  example  Is  obtained  by  considering  neurons  with 
"alterable  synapses"  or  "alterable  endbulbs"  of  the  following 
kind.  Each  neuron  may  have  besides  the  usual  endbulbs  also 
exeltatory  ones  which  are  not  effective  unless  at  some  previous 
time  the  neuron  having  the  endbulb  and  the  neuiron  to  which  the 
endbulb  Is  adjacent  were  simultaneously  fired.  If  a  neuron 
has  r  such  alterable  endbulbs.  It  Is  capable  of  2--  states, 
aceox^ing  as  It  Is  quiet  or  firing  and  according  to  which  of 
the  r  alterable  endbulbs  have  thus  far  been  made  effective. 

Many  other  possibilities  suggest  themselves. 

State ;  With  Input  cells  Ni,...,Nj^  and  Inner  cells 
with  respective  numbers  of  states  there 

It 

are  possible  at  a  given  moment  exactly  2—  *  £i*.*.*^  states  of 
the  entire  automaton.  We  can  consider  each  as  a  combination 

If 

of  an  external  state,  of  which  there  are  2—  possible,  and  an 
internal  state  of  which  there  are  £i*****Sjg  possible. 

The  law  by  which  the  states  of  the  Inner  cells  at  time  t 
are  determined  by  the  states  of  all  the  cells  at  time  t— 1  can 
be  given  by  specifying  to  each  of  the  complete  states  at  time 
t— 1  which  one  of  the  Inner  states  at  time  t  shall  succeed  It. 

Now,  Indeed,  there  Is  no  reason  for  our  general  theory 
why  we  cannot  consider  the  entire  aggregate  of  internal  cells 
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as  replaced  by  a  single  one  admitting  states.  This 

normalization  of  our  concept  of  a  finite  automaton  Is  always 
possible,  though  we  did  not  start  out  with  it,  because  we 
were  Interested  in  making  clear  the  application  to  such  automata 
as  a  McCulloch— Pitts  nerve  net,  where  the  cells  are  given  cer¬ 
tain  simple  properties  and  are  connected  in  a  certain  way. 

We  could  also  restrict  Ourselves  to  one  input  cell,  by 
scheduling  the  Inputs  on  the  k  original  input  cells  to  come  in 
successively  in  some  oi^er  on  the  new  one,  which  would  alter 
the  time  scale  so  that  k  moments  of  the  new  time  scale  correspond 


to  1  of  the  original.  Events  referring  to  the  new  time  scale 
could  then  be  interpreted  in  terms  of  the  original.  However, 
we  do  not  find  any  advantage  in  this  reduction  to  one  input 
neuron;  so  we  do  not  use  it. 

We  will  now  assume  that  time  starts  with  jb  =  1.  Say  we 

Jr 

call  the  states  a4,...,a_  where  r»2— ‘Si  '...*s  and  the 

r  —  — 

Internal  states  bi,  ...,b^  where  £  =  *'*'*Sja*  specify  that 

the  internal  state  at  time  t  =  1  be  bi. 


Under  this  assiimptlon,  the  state  at  time  t  =  £  is  a  function 
of  the  input  over  time  (including  the  value  of  £,  or  only 

this  when  k  =  0).  (Had  we  not  specified  the  Initial  state  as  bi 
the  state  at  time  £  would  be  a  ftonction  of  the  initial  state  also  .) 

So  each  of  the  states  ai,...,ap  corresponds  to  (or  repre¬ 
sents)  an  event,  vSiich  occurs  ending  at  time  £,  if  and  only  if 
the  input  over  the  time  l,...,p  is  one  which  results  in  that 


one  of  ai,...,aj,  being  the  state  at  time  £.  Thus,  the  automaton 
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can  know  about  the  past  (inclusive  of  the  present)  only  that 
it  falls  into  one  of  r  mutually  exclusive  classes  (possibly 
some  of  them  empty) . 

Similar  remarks  apply  to  representations  of  the  past  by 
an  internal  state  assumed  at  time  p+1,  or  by  a  property  of  the 
state  at  time  £,  or  of  the  internal  state  at  time  £fl.  For 
to  say  the  internal  state  at  time  p+1  is  b^  means  that  the  com¬ 
plete  state  at  time  p  was  one  of  certain  ones,  l.e.,  those 
which  are  succeeded  by  b^  under  the  law  determining  internal 
state.  So  then  the  past  falls  into  a  class  of  possible  pasts 
constituting  the  set  sum  of  the  classes  represented  by  those 
complete  states  at  time  £,  or  in  logical  terms  the  disjunction; 
fciffillarly^  for  properties  of  the  state  'similarly  also*; 
e.g.,  a  property  of  the  Internal  state  at  time  £ts  for  £  >  1, 
whenever  this  property  does  not  depend  on  the  input  over  time 
£+1, . . . ,£+£— 1) . 

What  sorts  of  events  can  be  represented?  The  question  is 
answered  by  the  following  theorem,  referring,  of  course,  to 
autcanata  started  in  state  b*.  Had  we  not  specified  the  initial 
state,  we  would  merely  add  (or  disjoin)  the  classes  corresponding 
thus  to  the  £  internal  states,  each  in  turn  as  Initial  state. 

Had  we  not  specified  past  time  to  be  finite,  the  state  at 
a  given  time  £  would  not  necessarily  be  determined  by  the  input. 
The  facts  in  this  case  (already  mentioned  in  Sect.  6.1  for 
MeCulloch-Fltts  nerve  nets)  are  dealt  with  in  Appendix  1. 

As  the  concept  of  input  is  the  same  as  in  Part  I,  we  can 
use  the  notion  df  "regular  event"  which  was  introduced  in  Sect.  7. 
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9.  Regularity  of  Representable  Events; 

Theorem  6:  For  any  <ginlte  automaton  (in  particular,  for 
a  MeCulloeh— Pitts  nerve  net)  started  at  a  given  time  t  =  1 
with  Internal  state  bi  at  that  time,  the  event  represented 
by  a  given  state  existing  at  time  £  Is  regular;  1 •£• ,  the 
automaton  assumes  that  state  at- time  p.  If  and  only  If  a  cer¬ 
tain  regular  event  occurs  ending  at  time  £. 

Proof;  Since  the  Initial  internal  state  Is  specified, 

Ic  / 

there  are  2—  initial  states  (the  results  of  combining  the 

It 

given  initial  Internal  state  with  each  of  the  2—  possible 
external  states  at  time  t  *  1)  from  which  the  automaton  could 
start  at  time  ^  =  1  to  reach  the  state  in  question  at  time 
t  =  £. 

So  if  we  can  show  that  the  automaton  can  start  from  a  given 

state  at  time  1  and  reach  a  given  state  at  time  £,  if  and  only 

if  a  certain  regular  event  occurs  ending  at  time  £,  then  the 

theorem  will  follow  by  taking  the  disjunction  of  2—  respective 

regular  events,  which  (by  Sect.  7.1)  is  itself  a  regular  event. 

/  \  ic 

Given  any  state  a  at  time  ;t— 1  (^—1  >1),  exactly  2—  states 
are  possible  at  time  t,  since  the  internal  part  of  the  state  at 
time  t  is  detennined  by  a,  and  the  external  part  can  happen  in 

Ir 

2—  ways. 

So  we  have  a  one-many  relationship  between  states.  Now 
invert  this  relation  and  consider  for  any  state  a  at  time  ^ 
what  states  at  time  ^—1  are  compatible  with  it  (there  may  be 
none,  one,  or  more  than  one);  say  a  is  in  relation  R  to  each 


of  these. 
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The  next  part  of  our  analysis  will  apply  to  any  binary 
relation  R  defined  on  a  given  set  ^f  r  objects  ai,...,a^  (calDed 
’‘states”),  whether  or  not  it  arises  in  the  manner  just  des¬ 
cribed. 

Consider  any  two  a  and  a  of  the  states,  not  necessarily 
distinct.  We  will  seek  a  characterization  of  the  strings  of 

dj^  is  a,  and  for  each 

i  (l  =  1,...,^— l)  is  in  the  relation  R  to  d^;  call  these 

strings  which  connect  a  to  a. 

Let  A  be  a  class  of  such  strings.  We  call  A  regular,  if 
A  can  be  described  by  an  expression  built  out  of  the  following 
operations  (chosen  in  analogy  to  the  definition  of  regular 
events  in  Sect.  7.1.) 


states  ds 


.  ..d,  for  which  dy  is  a. 


The  empty  set  and  the  unit  set  consisting  of  just  a^  for 
any  ^  are  regular.  If  A  and  B  are  regular,  so  is  their  sum 
which  we  write  AVb.  If  A  and  B  are  regular,  so  is  the  set, 
written  AB,  of  strings  obtained  by  writing  a  string  belonging 
to  A  just  left  of  a  string  belonging  to  B.  If  A  and  B  B.t*$ 
regular,  so  is  A*B  which  abbreviatts  '&  •  * (n  ^  0), 
i.e.,  the  sum  of  these  classes  for  ill  g  ^  G. 

Now  we  prove  the  lenHaa  by  induction  on  r  that  the  strings 
d^.  ..d^  connecting  a  to  U'  fam  m  regtilar  claii. 

Basis;  r  =*  1.  Then,  of  course,  a  is  a.  If  a  R  a  (i.e., 
if  not  a  R  a),  the  class  of  the  connecting  strings  is  simply 
the  unit  set  consisting  of  a  (as  string  of  length  1),  which  is 
regular.  If  a  R  a,  then  the  class  is  |a,  aa,  aaa,  . 
is  regular,  since  it  can  be  written  A*A  where  A  =  fa 
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Induction  step:  r  >  1. 

Case  1:  a  =  a.  In  this  ease  any  string  leading  from 

a  to  a  Is  of  the  form 

a  - a  - - ...  a  - — ^  a, 

no  a's  no  a's  no  a's 


Figure  28 


where  each  arrow  Is  either  empty  (this  being  possible  only  If 

a  R  a),  or  stands  for  a  string  without  a  In  It. 

Let  ei,...,e  be  the  states  e  such  that  a  R  e,  but  e  ^  a, 

£ 


and  fi,...,fj^  the  states  f  such  that  f  R  a  but  f  ^  a.  Now  any 
string  of  the  kind  represented  by  the  arrow  (when  the  arrow 
does  not  represent  the  absence’  of  a  string)  must  start  with 
one  df  ei,...,eg  and  end  with  one  of  fi,...,fj^.  For  each  pair 

e^i^fj,  by  the  hsrpothesis  of  the  induction,  the  class  of  the 


strings  leading  from  e^  to  f^^  (without  a  In  it)  is  regular. 

Say  are  these  regular  classes^  let  A  be 

If  a  R  a  and  the  B's  are  not  all  empty  (Subcase  l),  the  class 


of  possible  strings  a- - ^for  Pig.  28  is  AV  A(B,t  V  . . .  VB^j^)  I  If 


a  R  a  but  all  B*s  are  empty  (Subcase  ii),  it  is  A;  If  a  R  a  and 
the  B’s  are  not  all  empty  (Subcase  ill),  it  is  A(Bi  V  . . .  V  B^) ; 

and  If  a  R  a  and  the  B’s  are  all  empty  (Subcase  iv).  It  Is 
empty.  In  the  first  three  subcases,  let  G  denote  the  class  men¬ 
tioned,  which  is  non-empty  and  regular.  Now  in  these  subcases, 
the  class  of  strings  leading  from  a  to  a  (as  in  Fig.  28)  is 


C*A,  while  in  the  fourth  subcase  it  Is  simply  A. 
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a  /  a.  Now  we  have  instead  of  Fig.  28  the 


a - ^  ...  a - >  a  - - - ^ 

no  a ’ s  no  a ' 3  no  a ' s  or  a ' s 

a  —  ^  ...  a  ^  a 

no  a' s  or  a* s  no  a' s  or  a’ s 

Figure  29 

The  treatment  is  similar.  For  example,  in  the  case  the  classes 

of  strings  represented  by  "a - by  ”a - “  in 

no  a's  no  a* s  or  a' s 

the  middle,  and  by  ”a - "  at  the  right,  are  none 

no  a's  or  a’s 

of  them  empty,  call  them  C,  D,  and  E,  respectively,  the  class 
of  strings  for  Pig.  29  is  C*ADE*]r,  where 

So  the  lemma  is  proved.  Now  we  return  to  the  point  where 
we  were  in  the  proof  of  the  theorem.  We  wish  to  show  that  for 

fr  — 

given  state  a  at  time  £  and  each  of  2—  possible  states  a  at 

time  1,  that  a  holds  at  t  =  £  and  a  at  ^  =  1,  if  and  only  if 

a  certain  proper  event  (different  in  general  for  each  of  the 
k  —  \ 

2—  a's)  occupies  the  time  1  to  £. 

Now  by  the  lemma,  the  strings  of  states  which  can  lead 
f3?om  a  to  a  form  a  regular  class.  If  that  class  is  empty, 
then  the  event  is  the  Improper  one,  which  is  regular.  If  that 
class  is  not  empty,  consider  the  expression  for  the  class  as 
a  regular  class  of  strings*  We  build  a  corresponding  expression 
for  the  event  as  a  regular  event  by  translating  each  unit 
class  ^  (^"0^  each  state  a^)  into  the  definite  event  E^ 


Case  2; 

following: 

a - ^ 

no  a’s 
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of  length  1  which  occurs  at  time  £  exactly  If  the  Input  at 
time  £  Is  the  extei*nal  part  of  the  state  a^.  (If  k  =  0, 

Is  the  event  I  of  length  1  vrtilch  always  occurs,  having  no  other 
features.  However,  It  may  he  Initial  1,  that  Is,  E®*  The  only 

other  event  of  length  1  In  this  case  Is  the  Improper  one  T. ) 
Having  done  this,  then  the  operations  E\/Fj  EF,  and  E*F  for 
building  regular  events  parallel  those  aVb,  AB,  and  A*B  for 
building  regular  classes  of  Strings.  The  earliest  units 
(Sect.  7.5)  In  the  expressions  obtained  should  be  marked  as 
being  Initial. 

This  proves  the  theorem.  No  attempt  has  been  made  to 
consider.  In  this  proof,  how  simply  the  event  represented  by 
the  state  at  time  £  can  be  constructed  as  a  regular  event.  We 
have  worked  out  some  simple  Illustrations  In  which  very  compli¬ 
cated  expressions  stand  for  regular  events  capable  of  simple 
ones.  The  expressions  obtained  have  entirely  Initial  events 
as  earliest  iinlts,  and  are  built  of  units  of  length  1.  It  Is 
clear  that  In  most  examples  great  simplifications  can  be  obtained 
by  use  of  equivalences  (Sect.  7 .3)  i  t>nt  no  study  has  yet  been 
made  of  the  possibilities  for  proceeding  systematically  with 
such  simplifications,  or  of  rearranging  the  proof  of  Theorem  6 
to  come  out  directly  with  simpler  expressions  when  possible. 

The  study  of  the  structure  of  a  set  of  objects  ai,...,a^ 
under  a  relation  R,  which  Is  at  the  heart  of  the  above  proof, 
might  profitably  draw  on  some  algebraic  theory,  since  It  Is 
possible  (though  whether  profitable  or  not  we  do  not  know)  to 
see  the  situation  as  a  generalization  of  permutation  groups. 
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Corollary;  The  event  ending  at  time  £  represented  by 
each  of  the  following  Is  likewise  regular;  (a)  a  certain 
Internal  state  at  time  £+1.  (b)  a  property  of  the  state  at 

time  £.  (e)  a  property  of  the  internal  state  at  time  £+1. 

(d)  a  certain  Internal  state  at  time  £f£  for  a  given  s  >  1, 
when  this  does  not  depend  on  the  Input  over  £+1  ^  £+s— 1. 

(e)  a  property  of  the  Internal  state  at  time  p-fs  for  a  given 
£  >  1,  when  this  does  not  depend  on  the  Input  over  £+1  to 

£+ s— 1 . 

Proof;  As  remarked  at  the  end  of  Sect.  8,  each  of  these 
is  equivalent  to  one  of  certain  states  existing  at  time  p;  so 
the  event  represented  Is  a  dlsjvmctlon  of  the  regular  events 
given  by  the  theorem  for  the  latter,  and  hence  is  regular. 

This  corollary  brings  our  result  now  Into  correspondence 
(as  converse)  of  the  result  In  Sect.  7.  There  we  represented 
an  event  by  firing  a  certain  neuron  at  time  £+2.  This  Is  a 
property  of  the  internal  state  at  time  p+2,  since  it  means  the 
Internal  state  then  is  one  of  2—  different  ones  (according 
to  the  states  of  the  other  ra^l  Inner  neurons). 

Incidentally,  the  representations  In  Sect.  7  by  firing  a 
certain  neuron  at  time  £+2  are  equivalent  to  representations 
by  a  property  of  the  state  at  time  £,  namely  by  the  property 
which  those  states  at  time  £  share  which  will  lead  to  the  firing 
of  this  neuron  at  time  £+2. 

The  event  which  is  represented  by  a  state  a  Is  the  solu¬ 
tion  in  the  sense  in  which  McCulloch— Pitts  speak  of  ^solution" 
for  the  case  of  nerve  nets,  except  that  we  give  the  solution 
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for  a  given  internal  initial  state.  A  solution  without  pre¬ 
supposing  an  initial  internal  state  would  then  he  obtained  as 
a  disjunction  of  the  solutions  for  us  for  each  of  the 
(for  nerve  nets,  2^^)  Internal  states. 

Appendix  3  contains  an  example  of  an  event  which  cannot 
be  represented  in  a  finite  automaton. 

It  is,  of  course,  essential  for  our  arguments  here  that 
the  number  of  cells  or  parts  (xinder  our  first  definition  of 
a  finite  automaton)  and  the  nvimber  of  states  for  each,  be 
finite,  so  that  the  number  of  complete  states  is  fixed  in 
advance.  A  machine  of  Turing  (1937)  which  is  supplied  with  an 
unlimited  araotint  of  tape,  is  not  a  finite  automaton  in  our 
present  sense,  since,  although  in  its  operation  only  a  finite 
ntunber  of  squai*es  of  tape  are  printed  upon  at  any  time,  there 
is  no  preassigned  bovind  to  this  nximber. 

The  Turing  machine  could  be  thought  of  as  a  finite  auto¬ 
maton,  which  is  also  able  to  store  information  in  the  environ¬ 
ment  and  reach  for  it  later,  so  that  in  certain  cases  the  inputs 
are  identified  with  Inputs  at  earlier  times  or  with  states  of 
certain  inner  cells  at  earlier  times,  and  thus  the  present  input 
is  not  entirely  Independent  of  the  past.  Whether  this  compari¬ 
son  may  lead  to  any  useful  insights  into  Turing  machines,  or 
reciprocally  into  finite  automata,  remains  undetemlned. 
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APPENDIX  1;  DEFINITENESS  OP  EVENTS  REPRESENTABLE  IN  A  FINITE 
AUTOMATON  WITH  AN  INFINITE  PAST; 

Theorem  7 ;  Every  event  E  ending  at  time  p  repre sen table 
by  a  certain  state  existing  at  time  p  (or  by  one  of  the  other 
methods  listed  In  Corollary  Theorem  6)  In  a  finite  automaton 
with  an  Infinite  past  Is  definite. 

The  result  was  cited  In  Sect.  6.1.  The  notion  of  auto¬ 
maton  to  which  we  refer  Is  given  In  Sect.  8. 

Proof;  With  k  Input  cells,  the  complete  past  is  generated 

Ir 

by  choosing  between  a  finite  number  of  2—  possible  Inputs  at 

Ir 

time  t  =>  £,  then  between  a  finite  number  2—  at  time  t  =  p— 1, 
etc..,,  aul  infinitum. 

By  a  theorem  given  by  Brcuwsr  {1924}  and  aJ.??o  by  Xo&lg 

(1927),  if  for  each  infinite  past  (i.e.,  for  any  such  choice 
sequence)  it  is  determined  at  some  finite  stage  whether  the 
event  occurs  (ending  at  time  £)  or  not,  then  there  must  be  a 
number  N  >  0  such  that,  whether  the  event  occurs  or  not  is  known 
for  all  pasts  (l.e.,  all  choice  sequences)  from  only  that  part 
of  the  past  occupying  the  time  £-N  to  p.  In  this  ease  the  event 
would  be  definite  of  length  N+l.  (Brouwer's  proof  of  the  theorem 
Is  intended  for  readers  acquainted  with  the  Intultlonlstlc  set 
theory,  and  the  main  effort  in  his  proof  Is  to  demonstrate  the 
theorem  intuitionlstlcally. ) 

Now  we  show  that  indefinite  events  are  not  representable. 
Contraposigg  the  mentioned  theorem,  we  conclude  that  for  an 
Indefinite  event,  there  is  scwie  particular  infinite  past  such 
ithat  for  every  u  it  is  not  known  from  the  part  from  £-u  to  £ 

of  the  oa-v>t  wh-si.hor  evond  occur?:  or 
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Case  1;  The  event  E  does  not  occur  for  this  particular 
past.  Then  for  every  u  there  is  a  past  coinciding  with  the 
given  one  over  the  time  p— u  to  p  and  diverging  from  it  prior 

to  p—u.,  for  which  the  evemt  occurs. 

Mow  suppose  ifhe  representation  of  the  event  is  by  a  pro— 

perty  of  the  state  at  time  p;  say  the  notation  is  arranged  so 
that  the  states  which  have  this  property  are  and 

the  states  which  do  not  are  a  ...... ,a  . 

ri+1'  ’  r 

Now  consider  the  set  S  of  all  the  sequences  of  states 
dQd^dg  . . .  compatible  with  the  present  state  being  one  of 

•  *^1.^  i.e.,  dg  is  one  of  ^  each  has 

as  its  internal  part  that  which  is  detemined  by  d^^^^  There 

are  r-ri  choices  for  d^,  at  most  r  for  di  ,  at  most  r  for  dg,  etc» 
But  for  any  u  there  is  a  past  coinciding  with  the  given 
one  over  the  time  £-u  to  £,  and  diverging  from  it  before  that, 
along  which  the  event  does  not  occur.  Along  this  past,  any 
sequence  d^d^dg  . . .  must  belong  to  S  aiid  must  In  its  first 

u+1  choices  ^6  compatible  with  the  given  past  (as 

selected  above);  i.e.,  the  external  parts  of  d^,  d^,  . . . ,d^ 

must  be  the  inputs  over  the  last  u+1  moments  of  that  past. 

Now  by  Brouwer’s  theorem  (contraposed)  there  must  be  an 
infinite  sequence  dQd^dg...  in  the  set  S  which  is  compatible 


with  the  entire  given  past  (along  which  E  oeeurss,  but  from 
every  finite  segment  of  which  a  past  diverges  along  which  E 

does  net  occur) ,  .  ■ 
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But  is  one  of  the  states  a  ,  contrary  to 

u  £1 

our  assumption  that  E  Is  represented  hy  the  state  at  ^  =  p  being 

one  of  ai,,..,a  .  Thus,  E  cannot  be  so  represented.. 

If  ws  had  #®sumsd  simply  that  whenever  E  occurs,  th® 

present  state  must  be  one  of  ai,...,a^^,  the  above  considera¬ 
tions  show  that  there  must  also  be  examples  in  which  the  present , 
State  will  be  one  of  ai,...,a^^  without  E  having  occurred. 

Case  2;  The  event  E  does  not  occur  for  this  particular 
^ast.  The  reasoning  already  applied  gives  the  absurdity  of  E 
being  represented,  hence  of  E  itself  being  represented,  by  a 

property  of  the  state  at  t  p. 
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APPH&IX  2;  PRIMITIVE  RECURSIVENESS  OF  REGULAR  EVENTS; 

Theorem  8;  Every  regular  event  Is  primitive  recursive. 

The  terminology  in  the  theorem  is  from  the  theoi^y  of 
recursive  functions  and  predicates  as  developed  in  the  last  25 
years.  A  book  by  Peter  summarizes  the  theory,  also  a  book  by 
the  author  which  it  is  hoped  will  soon  be  in  print. 

The  fomulas  given  below  "place"  the  regular  events  in 
relation  to  number— theoretic  predicates  studied  in  the  theory 
of  recursive  functions.  Although  we  have  not  pursued  the  mat¬ 
ter  further  than  to  get  one  way  of  expressing  regular  events 
recursively,  possibly  useful  characterizations  may  be  obtain¬ 
able  in  this  direction. 

We  already  know  from  Sects.  5^  6.3,  and  7.1  that  number- 
theoretic  formulas  can  be  constructed  to  stand  for  definite 
events.  The  symbolism  required  can  be  seen  by  inspection  of 
the  examples  given.  Terms  ;^1,  £-2,  ...  are  used  only  when 
they  are  greater  than  0,  as  is  Insured  by  adding  £  >  ^ 

(Sect.  6.3)  or  now  sometimes  £  *  ^  (Sect.  7.1)  to  the  expres¬ 
sions  as  given  in  Sect.  5. 

The  range  of  the  variables  in  the  theory  of  recursive 
functions  is  customarily  0,  1,  2,  ...  rather  than  1,  2,  3,  ...  . 
To  avoid  having  to  reconstruct  the  notations  in  that  theory 
for  the  present  application,  let  us  in  this  appendix  suppose 
the  time  scale  for  events  starts  with  t  =  0  instead  of  t  =  1. 
Slight  changes  are  then  required  in  the  formulas  for  definite 
events.  Incidentally,  now  g— 1,  p— 2,  ...  are  used  in  the  sense 
of  £^1,  pf^  in  the  theory  of  recursive  functions. 
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We  shall  simultaneously  build  expressions  for  regular 
events  and  for  the  lengths  of  the  definite  events  of  the  class 
of  definite  events  which  we  use  In  characterizing  the  regular 
events.  More  precisely,  we  describe  to  each  regular  event  E 
expressions  A{p),  A(p,n) ,  ^ (p) ,  and  _y^(n)  such  that 

|e  has  occurred  ending  at  time  p|  ^  ^  ^  > 

where  for  each  n< (p ) ,  A(p,n)  describes  a  definite  event  of 
length  ^(n).  Here^(p)  Is  the  number  (>  l)  of  definite  events. 
In  the  occurrence  of  one  of  which  the  regular  event  consists. 
(Our€(£)  Is  not  necessarily  the  least  ntimber  of  such  definite 
events,  but  Is  the  number  we  use  In  our  constmiction.  Also, 
the^(p)  regular  events  need  not  all  be  different.) 

For  a  definite  event  used  as  a  unit  In  the  construction  ct 
a  regular  event,  6 (p)  *  1  (so  n  has  only  one  value  O); 
and  A(£)  Is  the  expression  already  mentioned  for  the  event; 

A (£,n)  can  be  simply  A(p) ,  or  A(£)  ^  n  «  0  If  we  wish  n  to  appear 
explicitly  In  this  ease;  whlle_y^(o)  =  ^  where  ^  Is  the  length 
of  the  definite  event  under  consideration. 

For  a  regular  event  of  the  form  P^G,  say  that  B(£,n), 

^  (p)  and  y^in)  are  the  ”E(p)"  and  'l/^n)''  for  F, 

and  C(p,n),  7^(£)  and  ^  (n)  are  those  for  G. 

Now  take 

A(p)  =  A(p,n) 

=(!a)n<  e(£)  {s<  Sf  (e)  2-  s(p.n^  v  ^(e.)  £ 

(the  scope  of  the  prefix  (ln)n<g(p)  in  the  last  being  A(£,n)), 
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where 


.  ,  1  >^{n)  if  n  <  ^(p), 

6  (p)  =  £(P)  +  77  (p) .  (n)  ^  J  (j  t  \\  ^  ^ 

y  ^  I  -  j^(p))  if*  E  >  /(p)* 

Let  a/t  =  the  quotient,  andy^(a,b)  =  the  remainder, 
when  an  integer  a  is  divided  by  a  positive  integer  b.  Note 
that  as  a  ranges  from  0  to  kb— 1  (b  /  0),  the  pair  of  quanti¬ 
ties  over  all  pairs  of  niambers  x,^.  with 

X  <  k,  y  <  b. 

Now  for  PG,  given  expressions  for  P  and  for  G  as  before 
by  the  hypothesis  of  the  induction,  we  have 


A(p)  S  (l2.)n<  e{p) 


/’(n,  7J(e)))  ]  . 


where 


6  (p)  *  ^  (£)  ?l  (p) 

y^(n)  =  >^(  [n/7?(p)J  )  +  ?  (/ (E>'^(p)))* 

For  the  remaining  case  we  use  the  funetion  (a)^  defined 

thus.  First  let  p^  =  (2^®  L-'th  prime  counting  2  as  the  0— t^  . 

{So  Eq  -  2,  pi  =  3,  £5  =  13»  etc.)  Now 

the  highest  power  of  p^  which  divides  a,  if  a  ^  0, 

0  if  a  =  0. 


(a)i 


Note  that  (a)^  is  also  0  if  a  51^  0,  but  does  not  divide  a. 
For  example,  28  «  2®*7;  so  (28)q=2,  (28)  1  »  0,  (28)2  =* 


0, 
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(28)3  =  1,  and  (28)^  =  O'  for  any  1  >  3* 

As  a  ranges  over  all  non-negative  integers,  (^)o^  •  *  *  ^  ^2;^in_i 

range  over  all  m— tuples  of  natural  numbers;  and,  in  fact,  as  a 

^0  — m  1 

ranges  over  0  to  * . . .  ’  or  beyond  the  functions 

max  ( (a)Q,b0) , . . .  ,max(  (a)jjj_^,b^^)  range  over  all  i^tuples 

^0*  •  •  •  for  Xq  ^  ^  1*  function  (a)^ 

could  have  been  used  in  place  of  |^/bj  and  in  treating 

the  preceding  case.) 

Now  say  E  is  F*G,  where  expressions  as  before  are  assumed 

already  constructed  for  P  and  for  G.  An  occurrence  of  E  is 

H  factors 

an  occurrence  of  PEG  for  some  u  >  0.  But  for  a 

given  p,  we  must  have  u  <  £,  since  P  and  G  are  each  of  length 
>  1.  So  now  we  have  for  E*F  the  following: 

A(p)  =  (^)n<  g(p)A(p,n) 

=  <ln)n<£(E)  ((nj^,  £)B(max((n)^,  |(e)-1), 

p  -  I  y'imax  ((n)  ,  / (£)  i  ))) 

£=1  — 

n  max  ((n)«,p) 

^  0  (»ax  ((n)^^  ((n)o,£)+l'WE)  -  D.  P 

Y^imax  ((n)j,  ^  (p)  1)))J  , 


where 

6(e) 


£4-  ^  (e)+  7?(e) 


since  each  n  which  would  be  wanted  is  of  the  form 
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2- 


2Li 

Pi 


P 

u+1 


9 


where 

u  <  p, 

<  /(p)  , 

w  <  ?^(P)  > 


and 

niax((n)-,p)  , 

"s-1  K(inax  ((n)g,^(£)  -1))  + 

§  -1))- 


It  will  be  seen  by  readers  familiar  with  recursive  function 
theory  that  ^  occurs  ending  at  time  ^  is  thus  primitive 
recursive  in  the  predicates  Ni  (^) , . . .  ,^(^)  giving  the  inputs 
over  time  t  =  0,...,£,  though,  of  course,  the  recursive 
expressions  are  complicated.  Also,  we  can  express  the  result 
by  saying  |e  occurs  ending  at  time  ^  is  primitive  recursive  in 
£  and  a  number  giving  in  code  form  the  combined  input  from 
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APPENDIX  5;  AN  EXAMPLE  OF  AN  E^T  ViHICH  IS  NOT  REPRESENTABLE 

(AHP  TTOEPORE  NOT  REGULAR),  THOUGH  IT  IS  PRIMITIVE 
RECURSIVE; 

Consider  the  event  E  referring  to  one  input  cell  N,  des¬ 
cribed  as  follows: 

N  fires  at  time  t  =  v®  for  every  v  such  that  v®  <  £,  and 


only  then. 

(This  is  primitive  recursive,  since  it  can  be  expressed 


thus ; 


(a) 


u<p 


No  nerve  net  or  finite  automaton  of  any  other  kind  can 
represent  this  event.  For  consider  an  automaton  with  states 

Q-l  f  •  •  *  y  3-^,  • 

Assume  given  a  representation  of  the  event  by  a  property 
of  the  automaton  at  time  jg;  l.e.,  we  assume  that  there  are 
states,  say  a4.,...,a^^  (ri  <  m),  such  that  at  time  p  the  state 
is  or  is  not  one  of  these,  according  as  the  event  has  occurred 
or;:  nott. 

We  shall  show  that  this  assumption  leads  to  absurdity. 
Son&ider  any  ainaber  s  such  that  2s  >  r  i . 

^  Say  that  N  fired  at  times  ^  1,  4,  9>***»£®  and 

wevsr’'t-hereaft§Fr'  “  - 

Then  E  oecuiPs  for  £  =  1,  2, . . ,  s,f£+l|^  —  1  and  fort 
no  greater  £. 

Consider  the  states  of  the  automaton  at  times  s®+l, 

...  .  These  must  all  be  from  the  list  ai,...,aj,. 
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However,  beginning  with  time  N  is  never  fired;  so 

the  external  state  is  constant.  Thus,  each  state  for  all  time 
thereafter  will  be  determined  by  the  immediately  preceding 
state.  Hence,  since  there  is  only  a  finite  number  of  possible 
states  aS,...,a^  ,  the  sequence  of  the  states  di,da,d3,... 
beginning  with  that  at  time  £®-i-l  is  ultimately  periodic.  For 
after  r  states  at  most,  a  state  must  be  taken  for  the  second 
time,  and  thereafter  the  states  since  the  first  occurrence  of 
that  one  must  repeat  themselves  cyclically. 

However,  during  the  time  s®+l, . . . ,£®+2^,  the  state  must 
be  one  of  ai,...,a^^,  since  the  event  occurs  for  these  values 

of  p;  and  hence,  since  2£  >  r  1 ,  the  period  must  already  have 

become  established  (i.e.,  the  first  repetition  in  dt,d2,d3,... 

must  already  have  occurred)  by  the  time  £®+2£.  Hence  the 

state  at  time  s®+2s+l  (=  (s+l)^)  is  one  of  ai,...,a  ,  contrary 

—  —  —  £1 , 

to  the  fact  that  the  event  has  not  occurred  ending  at  time 
p  =  (s+l)®. 

It  is  not  suggested  that  the  event  in  question  would  be 
of  any  biological  significance.  But  the  example  is  given  to 
show  the  mathematical  limitations  to  what  events  can  be  repre¬ 
sented.  Of  course,  by  Appendix  2  we  already  knew  that  events 
not  primitive  recursive  are  not  representable;  but  the  present 
example  is  much  simpler. 

Without  either  appendix,  one  would  not  expect  events 
lipase  verification  involves  the  completion  of  an  infinite  pro¬ 
cess  (these  being  non— recursive)  to  be  representable.  The 
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present  example  does  not  Involve  the  completion  of  an  Infinite 
process;  but  it  does  involve  the  completion  of  a  finite  process, 
which  as  p  varies  is  unbounded,  and  this  likewise  transcends 
the  capabilities  of  a  fixed  finite  automaton. 

bjc 
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